zoukankan      html  css  js  c++  java
  • hive遇到的问题以及解决办法

    hive java.lang.ClassNotFoundException: Class org.apache.hive.hcatalog.data.JsonSerDe not found

    hadoop    | Error: java.lang.RuntimeException: Error in configuring object
    hadoop    | 	at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:113)
    hadoop    | 	at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:79)
    hadoop    | 	at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:137)
    hadoop    | 	at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:455)
    hadoop    | 	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
    hadoop    | 	at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:177)
    hadoop    | 	at java.security.AccessController.doPrivileged(Native Method)
    hadoop    | 	at javax.security.auth.Subject.doAs(Subject.java:422)
    hadoop    | 	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1886)
    hadoop    | 	at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:171)
    hadoop    | Caused by: java.lang.reflect.InvocationTargetException
    hadoop    | 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    hadoop    | 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    hadoop    | 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    hadoop    | 	at java.lang.reflect.Method.invoke(Method.java:498)
    hadoop    | 	at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:110)
    hadoop    | 	... 9 more
    hadoop    | Caused by: java.lang.RuntimeException: Error in configuring object
    hadoop    | 	at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:113)
    hadoop    | 	at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:79)
    hadoop    | 	at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:137)
    hadoop    | 	at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:38)
    hadoop    | 	... 14 more
    hadoop    | Caused by: java.lang.reflect.InvocationTargetException
    hadoop    | 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    hadoop    | 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    hadoop    | 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    hadoop    | 	at java.lang.reflect.Method.invoke(Method.java:498)
    hadoop    | 	at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:110)
    hadoop    | 	... 17 more
    hadoop    | Caused by: java.lang.RuntimeException: Map operator initialization failed
    hadoop    | 	at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.configure(ExecMapper.java:137)
    hadoop    | 	... 22 more
    hadoop    | Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.ClassNotFoundException: Class org.apache.hive.hcatalog.data.JsonSerDe not found
    hadoop    | 	at org.apache.hadoop.hive.ql.exec.MapOperator.getConvertedOI(MapOperator.java:328)
    hadoop    | 	at org.apache.hadoop.hive.ql.exec.MapOperator.setChildren(MapOperator.java:420)
    hadoop    | 	at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.configure(ExecMapper.java:106)
    hadoop    | 	... 22 more
    hadoop    | Caused by: java.lang.ClassNotFoundException: Class org.apache.hive.hcatalog.data.JsonSerDe not found
    hadoop    | 	at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:2299)
    hadoop    | 	at org.apache.hadoop.hive.ql.plan.PartitionDesc.getDeserializer(PartitionDesc.java:177)
    hadoop    | 	at org.apache.hadoop.hive.ql.exec.MapOperator.getConvertedOI(MapOperator.java:295)
    hadoop    | 	... 24 more
    hadoop    | 
    hadoop    | 
    hadoop    | FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask
    hadoop    | MapReduce Jobs Launched: 
    hadoop    | Stage-Stage-1: Map: 1  Reduce: 1   HDFS Read: 0 HDFS Write: 0 FAIL
    hadoop    | Total MapReduce CPU Time Spent: 0 msec
    

    解决办法:

    方法一:添加add hive-hcatalog-core包

    ADD JAR /usr/local/hive/lib/hive-hcatalog-core-2.3.2.jar;

    方法二:Jar放入${HIVE_HOME}/auxlib目录(这个方法解决了我的问题)

    在${HIVE_HOME}建立一个auxlib目录,把hive-hcatalog-core.xxx.jar放到auxlib目录,重启hiverserver2,成功显示OK

  • 相关阅读:
    工单系统(帮助中心)
    理解RESTful架构
    trace显示不出东西
    thinkphp
    在一个元素中查找子元素
    阻止表单元素失去焦点
    RelativeLayout不能调用measure去直接测量子元素
    兼容加载Xml字符串
    IE下载时提示无法下载,重试后成功
    借用layer让弹层不限制在iframe内部
  • 原文地址:https://www.cnblogs.com/lfm601508022/p/9188819.html
Copyright © 2011-2022 走看看