zoukankan      html  css  js  c++  java
  • hive遇到的问题以及解决办法

    hive java.lang.ClassNotFoundException: Class org.apache.hive.hcatalog.data.JsonSerDe not found

    hadoop    | Error: java.lang.RuntimeException: Error in configuring object
    hadoop    | 	at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:113)
    hadoop    | 	at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:79)
    hadoop    | 	at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:137)
    hadoop    | 	at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:455)
    hadoop    | 	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
    hadoop    | 	at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:177)
    hadoop    | 	at java.security.AccessController.doPrivileged(Native Method)
    hadoop    | 	at javax.security.auth.Subject.doAs(Subject.java:422)
    hadoop    | 	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1886)
    hadoop    | 	at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:171)
    hadoop    | Caused by: java.lang.reflect.InvocationTargetException
    hadoop    | 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    hadoop    | 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    hadoop    | 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    hadoop    | 	at java.lang.reflect.Method.invoke(Method.java:498)
    hadoop    | 	at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:110)
    hadoop    | 	... 9 more
    hadoop    | Caused by: java.lang.RuntimeException: Error in configuring object
    hadoop    | 	at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:113)
    hadoop    | 	at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:79)
    hadoop    | 	at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:137)
    hadoop    | 	at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:38)
    hadoop    | 	... 14 more
    hadoop    | Caused by: java.lang.reflect.InvocationTargetException
    hadoop    | 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    hadoop    | 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    hadoop    | 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    hadoop    | 	at java.lang.reflect.Method.invoke(Method.java:498)
    hadoop    | 	at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:110)
    hadoop    | 	... 17 more
    hadoop    | Caused by: java.lang.RuntimeException: Map operator initialization failed
    hadoop    | 	at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.configure(ExecMapper.java:137)
    hadoop    | 	... 22 more
    hadoop    | Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.ClassNotFoundException: Class org.apache.hive.hcatalog.data.JsonSerDe not found
    hadoop    | 	at org.apache.hadoop.hive.ql.exec.MapOperator.getConvertedOI(MapOperator.java:328)
    hadoop    | 	at org.apache.hadoop.hive.ql.exec.MapOperator.setChildren(MapOperator.java:420)
    hadoop    | 	at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.configure(ExecMapper.java:106)
    hadoop    | 	... 22 more
    hadoop    | Caused by: java.lang.ClassNotFoundException: Class org.apache.hive.hcatalog.data.JsonSerDe not found
    hadoop    | 	at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:2299)
    hadoop    | 	at org.apache.hadoop.hive.ql.plan.PartitionDesc.getDeserializer(PartitionDesc.java:177)
    hadoop    | 	at org.apache.hadoop.hive.ql.exec.MapOperator.getConvertedOI(MapOperator.java:295)
    hadoop    | 	... 24 more
    hadoop    | 
    hadoop    | 
    hadoop    | FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask
    hadoop    | MapReduce Jobs Launched: 
    hadoop    | Stage-Stage-1: Map: 1  Reduce: 1   HDFS Read: 0 HDFS Write: 0 FAIL
    hadoop    | Total MapReduce CPU Time Spent: 0 msec
    

    解决办法:

    方法一:添加add hive-hcatalog-core包

    ADD JAR /usr/local/hive/lib/hive-hcatalog-core-2.3.2.jar;

    方法二:Jar放入${HIVE_HOME}/auxlib目录(这个方法解决了我的问题)

    在${HIVE_HOME}建立一个auxlib目录,把hive-hcatalog-core.xxx.jar放到auxlib目录,重启hiverserver2,成功显示OK

  • 相关阅读:
    随便写写
    mysql 快速插入100完毕 40秒
    存储过程 插入表数据 循环
    打开地图拖动位置获取经纬度 给父窗口传值
    Go源码共读计划
    源码读起来,Go源码共读计划
    清除centos所有命令记录
    删除django后台最近一个动作提示。
    自动延期pycharm插件,非常好用.
    pycharm中使用solidity插件 ,编写solidity以及在pycharm内进行编译。
  • 原文地址:https://www.cnblogs.com/lfm601508022/p/9188819.html
Copyright © 2011-2022 走看看