zoukankan      html  css  js  c++  java
  • Sqoop-MySQL导入hive时id为文本解决

    错误如下

    19/05/27 15:32:51 ERROR tool.ImportTool: Import failed: java.io.IOException: Generating splits for a textual index column allowed only in case of "-Dorg.apache.sqoop.splitter.allow_text_splitter=true" property passed as a parameter
        at org.apache.sqoop.mapreduce.db.DataDrivenDBInputFormat.getSplits(DataDrivenDBInputFormat.java:195)
        at org.apache.hadoop.mapreduce.JobSubmitter.writeNewSplits(JobSubmitter.java:310)
        at org.apache.hadoop.mapreduce.JobSubmitter.writeSplits(JobSubmitter.java:327)
        at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:200)
        at org.apache.hadoop.mapreduce.Job$11.run(Job.java:1570)
        at org.apache.hadoop.mapreduce.Job$11.run(Job.java:1567)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:422)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1685)
        at org.apache.hadoop.mapreduce.Job.submit(Job.java:1567)
        at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1588)
        at org.apache.sqoop.mapreduce.ImportJobBase.doSubmitJob(ImportJobBase.java:199)
        at org.apache.sqoop.mapreduce.ImportJobBase.runJob(ImportJobBase.java:172)
        at org.apache.sqoop.mapreduce.ImportJobBase.runImport(ImportJobBase.java:269)
        at org.apache.sqoop.manager.SqlManager.importTable(SqlManager.java:692)
        at org.apache.sqoop.manager.MySQLManager.importTable(MySQLManager.java:124)
        at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:526)
        at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:634)
        at org.apache.sqoop.Sqoop.run(Sqoop.java:145)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
        at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:181)
        at org.apache.sqoop.Sqoop.runTool(Sqoop.java:232)
        at org.apache.sqoop.Sqoop.runTool(Sqoop.java:241)
        at org.apache.sqoop.Sqoop.main(Sqoop.java:250)
    Caused by: Generating splits for a textual index column allowed only in case of "-Dorg.apache.sqoop.splitter.allow_text_splitter=true" property passed as a parameter
        at org.apache.sqoop.mapreduce.db.TextSplitter.split(TextSplitter.java:65)
        at org.apache.sqoop.mapreduce.db.DataDrivenDBInputFormat.getSplits(DataDrivenDBInputFormat.java:192)
        ... 23 more

    解决

    修改命令如下

    sqoop import "-Dorg.apache.sqoop.splitter.allow_text_splitter=true"
  • 相关阅读:
    P1908 逆序对
    P3834 【模板】可持久化线段树 1(主席树)
    BZOJ 4300: 绝世好题
    Codevs 2185【模板】最长公共上升子序列
    P1439 【模板】最长公共子序列
    P3865 【模板】ST表
    【转】良心的可持久化线段树教程
    Codevs 1299 切水果
    P3388 【模板】割点(割顶)&& 桥
    P3805 【模板】manacher算法
  • 原文地址:https://www.cnblogs.com/EnzoDin/p/10931031.html
Copyright © 2011-2022 走看看