zoukankan      html  css  js  c++  java
  • sqoop 测试 --hive-delims-replacement 参数

    在hive的官方文档中给的例子中字段分隔符是01,但是在他的API文档中--hive-delims-replacement ,--hive-drop-import-delims   参数中会处理的字符是x01,一个时8进制的1,一个是16进制的1,有这么一点差别,lz为了确认这两个是否一样故做了下面的实验

    Example:
    CREATE TABLE page_view(viewTime INT, userid BIGINT,
         page_url STRING, referrer_url STRING,
         ip STRING COMMENT 'IP Address of the User')
     COMMENT 'This is the page view table'
     PARTITIONED BY(dt STRING, country STRING)
     ROW FORMAT DELIMITED
       FIELDS TERMINATED BY '01'
    STORED AS SEQUENCEFILE;

    The above statement lets you create the same table as the previous table.

    In the previous examples the data is stored in <hive.metastore.warehouse.dir>/page_view. Specify a value for the key hive.metastore.warehouse.dir in the Hive config file hive-site.xml.

    --hive-delims-replacement <arg>             Replace Hive record x01

                                                   and row delimiters ( )

                                                   from imported string fields

                                                   with user-defined string

    --hive-drop-import-delims                   Drop Hive record x01 and

                                                   row delimiters ( ) from

                                                   imported string fields

    这两个参数不能一起用

    元数据(中间的方格是01,用java代码生成的)

    11 QQjyyh qwqwqw 1 1111 2017/10/15 23:27:48
    15 javajyyh 中文 2 1212 2017/10/15 23:39:57

    sqoop import --connect jdbc:oracle:thin:@MSI:1521/study --username luo --password Sys_20170929 --table TB_NEWS --fields-terminated-by "01" --lines-terminated-by "
    " --hive-import --hive-overwrite --null-string "" --null-non-string "" --fetch-size 1000 -m 3 --create-hive-table  --hive-table luoqi_test.TB_NEWS --delete-target-dir 

    导入之后hive的结构是

    11.0 QQ|jyyh qwqwqw 1 1111 2017-10-15 23:27:48.0
    15.0 java|jyyh 中文 2 1212 2017-10-15 23:39:57.0

    结果,'01'被替换成正确的字符

    结论,01可以被--hive-delims-replacement 参数替换

    导出将这个符号还原?

  • 相关阅读:
    梯度下降算法实现
    windows10下Anaconda的安装与tensorflow、opencv的安装与环境配置
    在github创建用户
    看完教材不明白的问题
    自我介绍
    HDU 1098(条件满足 数学)
    HDU 1097(m次幂的个位数 规律)
    HDU 1046(最短路径 **)
    HDU 1045(炮台安置 DFS)
    HDU 1034(传递糖果 模拟)
  • 原文地址:https://www.cnblogs.com/rocky-AGE-24/p/7675164.html
Copyright © 2011-2022 走看看