zoukankan      html  css  js  c++  java
  • Hive-java.lang.ClassNotFoundException: org.apache.hadoop.hive.contrib.serde2.MultiDelimitSerDe

    Task with the most failures(4): 
    -----
    Task ID:
      task_1555476136794_8201_m_000000
    
    URL:
      http://hadoop1:8088/taskdetails.jsp?jobid=job_1555476136794_8201&tipid=task_1555476136794_8201_m_000000
    -----
    Diagnostic Messages for this Task:
    Error: java.lang.RuntimeException: Error in configuring object
        at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:113)
        at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:79)
        at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:137)
        at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:462)
        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:349)
        at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:174)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:422)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1685)
        at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:168)
    Caused by: java.lang.reflect.InvocationTargetException
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:110)
        ... 9 more
    Caused by: java.lang.RuntimeException: Error in configuring object
        at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:113)
        at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:79)
        at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:137)
        at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:38)
        ... 14 more
    Caused by: java.lang.reflect.InvocationTargetException
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:110)
        ... 17 more
    Caused by: java.lang.RuntimeException: Map operator initialization failed
        at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.configure(ExecMapper.java:137)
        ... 22 more
    Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.RuntimeException: java.lang.ClassNotFoundException: org.apache.hadoop.hive.contrib.serde2.MultiDelimitSerDe
        at org.apache.hadoop.hive.ql.exec.FileSinkOperator.initializeOp(FileSinkOperator.java:453)
        at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:358)
        at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:546)
        at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:498)
        at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:368)
        at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:546)
        at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:498)
        at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:368)
        at org.apache.hadoop.hive.ql.exec.MapOperator.initializeMapOperator(MapOperator.java:441)
        at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.configure(ExecMapper.java:116)
        ... 22 more
    Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException: org.apache.hadoop.hive.contrib.serde2.MultiDelimitSerDe
        at org.apache.hadoop.hive.ql.plan.TableDesc.getDeserializerClass(TableDesc.java:75)
        at org.apache.hadoop.hive.ql.exec.FileSinkOperator.initializeOp(FileSinkOperator.java:369)
        ... 31 more
    Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.hive.contrib.serde2.MultiDelimitSerDe
        at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
        at java.lang.Class.forName0(Native Method)
        at java.lang.Class.forName(Class.java:348)
        at org.apache.hadoop.hive.ql.plan.TableDesc.getDeserializerClass(TableDesc.java:72)
        ... 32 more

    解决

    在 /opt/cloudera/parcels/CDH/lib/hive/lib路径下找到hive-contrib-2.1.1-cdh6.0.0.jar包,上传到HDFS,添加即可
    add jar hdfs://hadoop1:8020/common/jar/hive-contrib-2.1.1-cdh6.0.0.jar;

  • 相关阅读:
    分布式全局ID生成器设计
    对volatile不具有原子性的理解
    理解单链表的反转(java实现)
    Spring Boot 自定义kafka 消费者配置 ContainerFactory最佳实践
    Java 重载方法的匹配规则-含有变长参数方法的匹配
    为什么会产生jar包冲突,如何排查jar包冲突?
    SpringBoot 整合mongoDB并自定义连接池
    Java使用Optional与Stream来取代if判空逻辑(JDK8以上)
    多级树的深度优先遍历与广度优先遍历(Java实现)
    Maven pom.xml 全配置(一)常用配置
  • 原文地址:https://www.cnblogs.com/EnzoDin/p/10930624.html
Copyright © 2011-2022 走看看