zoukankan      html  css  js  c++  java
  • sqoop 测试 --hive-delims-replacement 参数

    在hive的官方文档中给的例子中字段分隔符是01,但是在他的API文档中--hive-delims-replacement ,--hive-drop-import-delims   参数中会处理的字符是x01,一个时8进制的1,一个是16进制的1,有这么一点差别,lz为了确认这两个是否一样故做了下面的实验

    Example:
    CREATE TABLE page_view(viewTime INT, userid BIGINT,
         page_url STRING, referrer_url STRING,
         ip STRING COMMENT 'IP Address of the User')
     COMMENT 'This is the page view table'
     PARTITIONED BY(dt STRING, country STRING)
     ROW FORMAT DELIMITED
       FIELDS TERMINATED BY '01'
    STORED AS SEQUENCEFILE;

    The above statement lets you create the same table as the previous table.

    In the previous examples the data is stored in <hive.metastore.warehouse.dir>/page_view. Specify a value for the key hive.metastore.warehouse.dir in the Hive config file hive-site.xml.

    --hive-delims-replacement <arg>             Replace Hive record x01

                                                   and row delimiters ( )

                                                   from imported string fields

                                                   with user-defined string

    --hive-drop-import-delims                   Drop Hive record x01 and

                                                   row delimiters ( ) from

                                                   imported string fields

    这两个参数不能一起用

    元数据(中间的方格是01,用java代码生成的)

    11 QQjyyh qwqwqw 1 1111 2017/10/15 23:27:48
    15 javajyyh 中文 2 1212 2017/10/15 23:39:57

    sqoop import --connect jdbc:oracle:thin:@MSI:1521/study --username luo --password Sys_20170929 --table TB_NEWS --fields-terminated-by "01" --lines-terminated-by "
    " --hive-import --hive-overwrite --null-string "" --null-non-string "" --fetch-size 1000 -m 3 --create-hive-table  --hive-table luoqi_test.TB_NEWS --delete-target-dir 

    导入之后hive的结构是

    11.0 QQ|jyyh qwqwqw 1 1111 2017-10-15 23:27:48.0
    15.0 java|jyyh 中文 2 1212 2017-10-15 23:39:57.0

    结果,'01'被替换成正确的字符

    结论,01可以被--hive-delims-replacement 参数替换

    导出将这个符号还原?

  • 相关阅读:
    IE下判断IE版本的语句...[if lte IE 6]……[endif]
    “浏览器模式”和“文档模式”
    IoC框架---通俗概述
    Castle IOC容器组件生命周期管理
    Castle学习笔记----初探IOC容器
    Castle IOC容器内幕故事(下)
    Castle IOC容器内幕故事(上)
    Castle IOC容器构建配置详解(二)
    javascript属性一览
    javascript addEventListener方法
  • 原文地址:https://www.cnblogs.com/rocky-AGE-24/p/7675164.html
Copyright © 2011-2022 走看看