zoukankan      html  css  js  c++  java
  • 解决Hue/hiveserver2报错:java.io.IOException: Job status not available

    sql是:select count(distinct col) from db.table;

    排查过程中遇到过几个不同的报错:

    1. beeline -u jdbc:hive2://0.0.0.0:10000 -e "select count(distinct col) from db.table;"

    INFO  : Kill Command = /usr/lib/hadoop/bin/hadoop job  -kill job_1494385775332_0822
    ERROR : Ended Job = job_1494385775332_0822 with exception 'java.io.IOException(Job status not available )'
    java.io.IOException: Job status not available 
            at org.apache.hadoop.mapreduce.Job.updateStatus(Job.java:333)
            at org.apache.hadoop.mapreduce.Job.getJobState(Job.java:358)
            at org.apache.hadoop.mapred.JobClient$NetworkedJob.getJobState(JobClient.java:295)
            at org.apache.hadoop.hive.ql.exec.mr.HadoopJobExecHelper.progress(HadoopJobExecHelper.java:244)
            at org.apache.hadoop.hive.ql.exec.mr.HadoopJobExecHelper.progress(HadoopJobExecHelper.java:549)
            at org.apache.hadoop.hive.ql.exec.mr.ExecDriver.execute(ExecDriver.java:437)
            at org.apache.hadoop.hive.ql.exec.mr.MapRedTask.execute(MapRedTask.java:137)
            at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:160)
            at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:88)
            at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1653)
    ....
    Error: Error while processing statement: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask (state=08S01,code=1)
    Diagnostics: 
    Staging dir does not exist /mr-history/anonymous/.staging           

    2. beeline -u jdbc:hive2://0.0.0.0:10000 -n userxxx -e "select count(distinct col) from db.table;"

    ERROR : Ended Job = job_1494385775332_0823 with exception 'java.io.IOException(java.io.IOException: Unknown Job job_1494385775332_0823
            at org.apache.hadoop.mapreduce.v2.hs.HistoryClientService$HSClientProtocolHandler.verifyAndGetJob(HistoryClientService.java:218)
            at org.apache.hadoop.mapreduce.v2.hs.HistoryClientService$HSClientProtocolHandler.getTaskAttemptCompletionEvents(HistoryClientService.java:281)
            at org.apache.hadoop.mapreduce.v2.api.impl.pb.service.MRClientProtocolPBServiceImpl.getTaskAttemptCompletionEvents(MRClientProtocolPBServiceImpl.java:173)
            at org.apache.hadoop.yarn.proto.MRClientProtocol$MRClientProtocolService$2.callBlockingMethod(MRClientProtocol.java:283)
            at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:616)
            at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:969)
            at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2049)
            at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2045)
            at java.security.AccessController.doPrivileged(Native Method)
            at javax.security.auth.Subject.doAs(Subject.java:422)
            at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
            at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2043)
    )'
    java.io.IOException: java.io.IOException: Unknown Job job_1494385775332_0823
    Diagnostics: 
    Job init failed : org.apache.hadoop.yarn.exceptions.YarnRuntimeException: java.io.FileNotFoundException: File does not exist: hdfs://xxx/mr-history/userxxx/.staging/job_1494385775332_0823/job.splitmetainfo   

    使用不同的用户执行还是报一样的错误。

    3. hive -e "select count(distinct col) from db.table;" 却能执行成功

    解决:

    根据第二步Diagnostics的报错信息搜索到一篇文章:https://community.hortonworks.com/questions/17489/job-init-fail-job-splitmetainfo-file-does-not-exis.html

    对比了hive和hs2中此属性的值,果然不一样.hive上的值和配置文件中的一样,hs2不一样,在hs2中执行hql前设置为正确值后问题解决。询问了运维同事,这个属性近期刚改,改完后hiveserver2没有重启,导致使用beeline客户端或Hue时报此错。

    解决方法:

    1. 临时性:hql中显示纠正此属性值:set yarn.app.mapreduce.am.staging-dir=xxx;

    2. 重启HS2

  • 相关阅读:
    关于连通性问题的Tarjan算法暂结
    【BZOJ 3925】[Zjoi2015]地震后的幻想乡 期望概率dp+状态压缩+图论知识+组合数学
    Kruskal算法及其类似原理的应用——【BZOJ 3654】tree&&【BZOJ 3624】[Apio2008]免费道路
    【NOIP模拟赛】Drink 二维链表+模拟
    【BZOJ 2957】楼房重建&&Codechef COT5 Count on a Treap&&【NOIP模拟赛】Weed 线段树的分治维护
    【BZOJ 4198】[Noi2015]荷马史诗 哈夫曼编码
    【NOIP模拟赛】chess 建图+spfa统计方案数
    【TMD模拟赛】上低音号 链表
    【TMD模拟赛】黄金拼图 Cao
    【BZOJ 4007】[JLOI2015]战争调度 DP+搜索+状压
  • 原文地址:https://www.cnblogs.com/aprilrain/p/6916222.html
Copyright © 2011-2022 走看看