zoukankan      html  css  js  c++  java
  • sqoop 测试 --hive-delims-replacement 参数

    在hive的官方文档中给的例子中字段分隔符是01,但是在他的API文档中--hive-delims-replacement ,--hive-drop-import-delims   参数中会处理的字符是x01,一个时8进制的1,一个是16进制的1,有这么一点差别,lz为了确认这两个是否一样故做了下面的实验

    Example:
    CREATE TABLE page_view(viewTime INT, userid BIGINT,
         page_url STRING, referrer_url STRING,
         ip STRING COMMENT 'IP Address of the User')
     COMMENT 'This is the page view table'
     PARTITIONED BY(dt STRING, country STRING)
     ROW FORMAT DELIMITED
       FIELDS TERMINATED BY '01'
    STORED AS SEQUENCEFILE;

    The above statement lets you create the same table as the previous table.

    In the previous examples the data is stored in <hive.metastore.warehouse.dir>/page_view. Specify a value for the key hive.metastore.warehouse.dir in the Hive config file hive-site.xml.

    --hive-delims-replacement <arg>             Replace Hive record x01

                                                   and row delimiters ( )

                                                   from imported string fields

                                                   with user-defined string

    --hive-drop-import-delims                   Drop Hive record x01 and

                                                   row delimiters ( ) from

                                                   imported string fields

    这两个参数不能一起用

    元数据(中间的方格是01,用java代码生成的)

    11 QQjyyh qwqwqw 1 1111 2017/10/15 23:27:48
    15 javajyyh 中文 2 1212 2017/10/15 23:39:57

    sqoop import --connect jdbc:oracle:thin:@MSI:1521/study --username luo --password Sys_20170929 --table TB_NEWS --fields-terminated-by "01" --lines-terminated-by "
    " --hive-import --hive-overwrite --null-string "" --null-non-string "" --fetch-size 1000 -m 3 --create-hive-table  --hive-table luoqi_test.TB_NEWS --delete-target-dir 

    导入之后hive的结构是

    11.0 QQ|jyyh qwqwqw 1 1111 2017-10-15 23:27:48.0
    15.0 java|jyyh 中文 2 1212 2017-10-15 23:39:57.0

    结果,'01'被替换成正确的字符

    结论,01可以被--hive-delims-replacement 参数替换

    导出将这个符号还原?

  • 相关阅读:
    (转)Linux: su sudo sudoer
    unsigned 和 signed
    (转)原码, 反码, 补码 详解
    判断两个链表是否相交
    源代码里面看不到页面展示的数据?原因何在
    python 小技巧 防止SSL报错信息
    python爬虫之一---------豆瓣妹子图
    服务端性能测试 TPS
    H5性能测试学习
    App测试中 ----------------Android和IOS测试区别
  • 原文地址:https://www.cnblogs.com/rocky-AGE-24/p/7675164.html
Copyright © 2011-2022 走看看