zoukankan      html  css  js  c++  java
  • Hive-java.lang.ClassNotFoundException: org.apache.hadoop.hive.contrib.serde2.MultiDelimitSerDe

    Task with the most failures(4): 
    -----
    Task ID:
      task_1555476136794_8201_m_000000
    
    URL:
      http://hadoop1:8088/taskdetails.jsp?jobid=job_1555476136794_8201&tipid=task_1555476136794_8201_m_000000
    -----
    Diagnostic Messages for this Task:
    Error: java.lang.RuntimeException: Error in configuring object
        at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:113)
        at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:79)
        at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:137)
        at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:462)
        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:349)
        at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:174)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:422)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1685)
        at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:168)
    Caused by: java.lang.reflect.InvocationTargetException
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:110)
        ... 9 more
    Caused by: java.lang.RuntimeException: Error in configuring object
        at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:113)
        at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:79)
        at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:137)
        at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:38)
        ... 14 more
    Caused by: java.lang.reflect.InvocationTargetException
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:110)
        ... 17 more
    Caused by: java.lang.RuntimeException: Map operator initialization failed
        at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.configure(ExecMapper.java:137)
        ... 22 more
    Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.RuntimeException: java.lang.ClassNotFoundException: org.apache.hadoop.hive.contrib.serde2.MultiDelimitSerDe
        at org.apache.hadoop.hive.ql.exec.FileSinkOperator.initializeOp(FileSinkOperator.java:453)
        at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:358)
        at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:546)
        at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:498)
        at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:368)
        at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:546)
        at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:498)
        at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:368)
        at org.apache.hadoop.hive.ql.exec.MapOperator.initializeMapOperator(MapOperator.java:441)
        at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.configure(ExecMapper.java:116)
        ... 22 more
    Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException: org.apache.hadoop.hive.contrib.serde2.MultiDelimitSerDe
        at org.apache.hadoop.hive.ql.plan.TableDesc.getDeserializerClass(TableDesc.java:75)
        at org.apache.hadoop.hive.ql.exec.FileSinkOperator.initializeOp(FileSinkOperator.java:369)
        ... 31 more
    Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.hive.contrib.serde2.MultiDelimitSerDe
        at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
        at java.lang.Class.forName0(Native Method)
        at java.lang.Class.forName(Class.java:348)
        at org.apache.hadoop.hive.ql.plan.TableDesc.getDeserializerClass(TableDesc.java:72)
        ... 32 more

    解决

    在 /opt/cloudera/parcels/CDH/lib/hive/lib路径下找到hive-contrib-2.1.1-cdh6.0.0.jar包,上传到HDFS,添加即可
    add jar hdfs://hadoop1:8020/common/jar/hive-contrib-2.1.1-cdh6.0.0.jar;

  • 相关阅读:
    组合算法实现
    Memcached 和 Redis 分布式锁方案
    CLR 内存分配和垃圾收集 GC
    Windbg 的使用和常用命令
    Geohash 算法学习
    经纬度计算
    Windbg 分析CPU上涨
    Windbg 分析内存上涨
    django基于存储在前端的token用户认证
    非常详细的Django使用Token(转)
  • 原文地址:https://www.cnblogs.com/EnzoDin/p/10930624.html
Copyright © 2011-2022 走看看