zoukankan      html  css  js  c++  java
  • troubleshooting-When importing query results in parallel, you must specify --split-by.

    原因分析

    -m 4 导数命令中map task number=4,当-m 设置的值大于1时,split-by必须设置字段(需要是 int 类型的字段),如果不是 int类型的字段,则需要加上参数
    -Dorg.apache.sqoop.splitter.allow_text_splitter=true
    例子:
    sqoop import -Dorg.apache.sqoop.splitter.allow_text_splitter=true
    --connect ${conn_str}
    --username ${db_username}
    --password ${db_password}

    解决办法

    并行导入数据需要加上 --split-by columnName 属性。

    异常日志

    18/09/17 14:23:58 INFO sqoop.Sqoop: Running Sqoop version: 1.4.7-cdh6.0.0
    18/09/17 14:23:58 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
    When importing query results in parallel, you must specify --split-by.
    Try --help for usage instructions.
    SLF4J: Class path contains multiple SLF4J bindings.
    SLF4J: Found binding in [jar:file:/opt/cloudera/parcels/CDH-6.0.0-1.cdh6.0.0.p0.537114/jars/log4j-slf4j-impl-2.8.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
    SLF4J: Found binding in [jar:file:/opt/cloudera/parcels/CDH-6.0.0-1.cdh6.0.0.p0.537114/jars/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
    SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
    SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
    FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. Table not found user_base_delta

  • 相关阅读:
    用C#实现宽带重新拨号
    CALLBACK FUNCTION 回调函数
    编译程序 VS 解释程序
    《围城》读书笔记
    鼠标点击器
    工作与找工作的日子
    Windows 7下VS2003的查找无响应问题
    收藏几句关于程序的名言
    static知识小结
    如何定义和实现一个类的成员函数为回调函数(转)
  • 原文地址:https://www.cnblogs.com/chwilliam85/p/9693268.html
Copyright © 2011-2022 走看看