zoukankan      html  css  js  c++  java
  • hive遇到的问题以及解决办法

    hive java.lang.ClassNotFoundException: Class org.apache.hive.hcatalog.data.JsonSerDe not found

    hadoop    | Error: java.lang.RuntimeException: Error in configuring object
    hadoop    | 	at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:113)
    hadoop    | 	at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:79)
    hadoop    | 	at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:137)
    hadoop    | 	at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:455)
    hadoop    | 	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
    hadoop    | 	at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:177)
    hadoop    | 	at java.security.AccessController.doPrivileged(Native Method)
    hadoop    | 	at javax.security.auth.Subject.doAs(Subject.java:422)
    hadoop    | 	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1886)
    hadoop    | 	at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:171)
    hadoop    | Caused by: java.lang.reflect.InvocationTargetException
    hadoop    | 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    hadoop    | 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    hadoop    | 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    hadoop    | 	at java.lang.reflect.Method.invoke(Method.java:498)
    hadoop    | 	at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:110)
    hadoop    | 	... 9 more
    hadoop    | Caused by: java.lang.RuntimeException: Error in configuring object
    hadoop    | 	at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:113)
    hadoop    | 	at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:79)
    hadoop    | 	at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:137)
    hadoop    | 	at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:38)
    hadoop    | 	... 14 more
    hadoop    | Caused by: java.lang.reflect.InvocationTargetException
    hadoop    | 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    hadoop    | 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    hadoop    | 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    hadoop    | 	at java.lang.reflect.Method.invoke(Method.java:498)
    hadoop    | 	at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:110)
    hadoop    | 	... 17 more
    hadoop    | Caused by: java.lang.RuntimeException: Map operator initialization failed
    hadoop    | 	at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.configure(ExecMapper.java:137)
    hadoop    | 	... 22 more
    hadoop    | Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.ClassNotFoundException: Class org.apache.hive.hcatalog.data.JsonSerDe not found
    hadoop    | 	at org.apache.hadoop.hive.ql.exec.MapOperator.getConvertedOI(MapOperator.java:328)
    hadoop    | 	at org.apache.hadoop.hive.ql.exec.MapOperator.setChildren(MapOperator.java:420)
    hadoop    | 	at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.configure(ExecMapper.java:106)
    hadoop    | 	... 22 more
    hadoop    | Caused by: java.lang.ClassNotFoundException: Class org.apache.hive.hcatalog.data.JsonSerDe not found
    hadoop    | 	at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:2299)
    hadoop    | 	at org.apache.hadoop.hive.ql.plan.PartitionDesc.getDeserializer(PartitionDesc.java:177)
    hadoop    | 	at org.apache.hadoop.hive.ql.exec.MapOperator.getConvertedOI(MapOperator.java:295)
    hadoop    | 	... 24 more
    hadoop    | 
    hadoop    | 
    hadoop    | FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask
    hadoop    | MapReduce Jobs Launched: 
    hadoop    | Stage-Stage-1: Map: 1  Reduce: 1   HDFS Read: 0 HDFS Write: 0 FAIL
    hadoop    | Total MapReduce CPU Time Spent: 0 msec
    

    解决办法:

    方法一:添加add hive-hcatalog-core包

    ADD JAR /usr/local/hive/lib/hive-hcatalog-core-2.3.2.jar;

    方法二:Jar放入${HIVE_HOME}/auxlib目录(这个方法解决了我的问题)

    在${HIVE_HOME}建立一个auxlib目录,把hive-hcatalog-core.xxx.jar放到auxlib目录,重启hiverserver2,成功显示OK

  • 相关阅读:
    Python分析网页中的<a>标签
    Python3.X爬虫
    如何使你的窗体样式固定,不因系统设定而变化
    C# 窗体内有子控件时鼠标检测
    VS新建项目工具箱图标丢失问题
    在c#中使用bitblt显示图片
    DotNetBar 中Ribbon汉化
    汉化DotNetBar中控件的系统文本
    castle动态代理的使用
    FastReport 套打全攻略
  • 原文地址:https://www.cnblogs.com/lfm601508022/p/9188819.html
Copyright © 2011-2022 走看看