zoukankan      html  css  js  c++  java
  • sqoop 测试 --hive-delims-replacement 参数

    在hive的官方文档中给的例子中字段分隔符是01,但是在他的API文档中--hive-delims-replacement ,--hive-drop-import-delims   参数中会处理的字符是x01,一个时8进制的1,一个是16进制的1,有这么一点差别,lz为了确认这两个是否一样故做了下面的实验

    Example:
    CREATE TABLE page_view(viewTime INT, userid BIGINT,
         page_url STRING, referrer_url STRING,
         ip STRING COMMENT 'IP Address of the User')
     COMMENT 'This is the page view table'
     PARTITIONED BY(dt STRING, country STRING)
     ROW FORMAT DELIMITED
       FIELDS TERMINATED BY '01'
    STORED AS SEQUENCEFILE;

    The above statement lets you create the same table as the previous table.

    In the previous examples the data is stored in <hive.metastore.warehouse.dir>/page_view. Specify a value for the key hive.metastore.warehouse.dir in the Hive config file hive-site.xml.

    --hive-delims-replacement <arg>             Replace Hive record x01

                                                   and row delimiters ( )

                                                   from imported string fields

                                                   with user-defined string

    --hive-drop-import-delims                   Drop Hive record x01 and

                                                   row delimiters ( ) from

                                                   imported string fields

    这两个参数不能一起用

    元数据(中间的方格是01,用java代码生成的)

    11 QQjyyh qwqwqw 1 1111 2017/10/15 23:27:48
    15 javajyyh 中文 2 1212 2017/10/15 23:39:57

    sqoop import --connect jdbc:oracle:thin:@MSI:1521/study --username luo --password Sys_20170929 --table TB_NEWS --fields-terminated-by "01" --lines-terminated-by "
    " --hive-import --hive-overwrite --null-string "" --null-non-string "" --fetch-size 1000 -m 3 --create-hive-table  --hive-table luoqi_test.TB_NEWS --delete-target-dir 

    导入之后hive的结构是

    11.0 QQ|jyyh qwqwqw 1 1111 2017-10-15 23:27:48.0
    15.0 java|jyyh 中文 2 1212 2017-10-15 23:39:57.0

    结果,'01'被替换成正确的字符

    结论,01可以被--hive-delims-replacement 参数替换

    导出将这个符号还原?

  • 相关阅读:
    socket注意
    PCM音频文件编码
    题外:分类篇(音乐风格分类)基于BP神经网络
    MFCC特征提取过程详解
    语音信号分析
    k-means聚类
    c++关键字详解
    vs中项目解决方案和项目的关系
    条件编译#ifdef 和#endif
    c++快捷键
  • 原文地址:https://www.cnblogs.com/rocky-AGE-24/p/7675164.html
Copyright © 2011-2022 走看看