zoukankan      html  css  js  c++  java
  • 【原创】大叔经验分享(46)用户提交任务到yarn报错

    用户提交任务到yarn时有可能遇到下面的错误:

    1)

    Requested user anything is not whitelisted and has id 980,which is below the minimum allowed 1000

    这是因为yarn中配置min.user.id=1000,yarn认为id小于1000的是超级用户,yarn禁止超级用户提交任务;

    Each account must have a user ID that is greater than or equal to 1000. In the /etc/hadoop/conf/taskcontroller.cfg file, the default setting for the banned.users property is mapred, hdfs, and bin to prevent jobs from being submitted via those user accounts. The default setting for the min.user.id property is 1000 to prevent jobs from being submitted with a user ID less than 1000, which are conventionally Unix super users.

    改为更小即可

    然后重启yarn

    2)

    Requested user hdfs is banned

    这是因为yarn中配置banned.users=hdfs,yarn,mapred,bin,会阻止这几个用户提交任务

    the default setting for the banned.users property is hdfs, yarn, mapred, and bin to prevent jobs from being submitted via those user accounts.

    3)其他组件比如oozie无法获得提交到yarn上的application的状态

    2019-04-01 11:22:23,528 WARN org.apache.oozie.command.wf.ActionCheckXCommand: SERVER[cdp-test-server-04.bj] USER[impala] GROUP[-] TOKEN[] APP[test_sh_wf] JOB[0000000-190330171215585-oozie-oozi-W] ACTION[0000000-190330171215585-oozie-oozi-W@test_sh_action] Exception while executing check(). Error Code [JA017], Message[JA017: Could not lookup launched hadoop Job ID [job_1553937093273_0007] which was associated with  action [0000000-190330171215585-oozie-oozi-W@test_sh_action].  Failing this action!]
    org.apache.oozie.action.ActionExecutorException: JA017: Could not lookup launched hadoop Job ID [job_1553937093273_0007] which was associated with  action [0000000-190330171215585-oozie-oozi-W@test_sh_action].  Failing this action!
            at org.apache.oozie.action.hadoop.JavaActionExecutor.check(JavaActionExecutor.java:1507)
            at org.apache.oozie.command.wf.ActionCheckXCommand.execute(ActionCheckXCommand.java:182)
            at org.apache.oozie.command.wf.ActionCheckXCommand.execute(ActionCheckXCommand.java:56)
            at org.apache.oozie.command.XCommand.call(XCommand.java:286)
            at java.util.concurrent.FutureTask.run(FutureTask.java:266)
            at org.apache.oozie.service.CallableQueueService$CallableWrapper.run(CallableQueueService.java:179)
            at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
            at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
            at java.lang.Thread.run(Thread.java:748)

    此时请检查jobhistory server,如果jobhistory server页面中没有application,也可以通过curl查看:

    # curl http:/localhost:19888/jobhistory -v
    ...
                <script type="text/javascript">
                  var jobsTableData=[
    ]
                </script>
    ...

    正常来说这个数组应该存放很多application,这个问题需要检查hdfs上/user/history及子目录是否存在以及权限

    $ hdfs dfs -mkdir /user/history
    $ hdfs dfs -chmod 777 /user/history

    正常情况下jobhistory会从hdfs以下目录加载应用信息

    /user/history/done/$year/$month/$day/
    /user/history/done_intermediate/$user/

    4)从jobhistory server查看application时报错

    2019-04-01 16:04:16,115 WARN org.apache.oozie.command.wf.ActionCheckXCommand: SERVER[cdp-test-server-04.bj] USER[impala] GROUP[-] TOKEN[] APP[test_sh_wf] JOB[0000008-190330171215585-oozie-oozi-W] ACTION[0000008-190330171215585-oozie-oozi-W@test_sh_action] Exception while executing check(). Error Code [JA009], Message[JA009: org.apache.hadoop.yarn.exceptions.YarnRuntimeException: Could not load history file hdfs://hdfs:8020/user/history/done_intermediate/impala/job_1553937093273_0018-1554105820398-impala-oozie%3Alauncher%3AT%3Dshell%3AW%3Dtest_sh_wf%3AA%3Dtest_sh_acti-1554105833651-1-0-SUCCEEDED-root.users.impala-1554105825680.jhist
    ...
    Caused by: org.apache.hadoop.security.AccessControlException: Permission denied: user=mapred, access=READ, inode="/user/history/done_intermediate/impala/job_1553937093273_0018-1554105820398-impala-oozie%3Alauncher%3AT%3Dshell%3AW%3Dtest_sh_wf%3AA%3Dtest_sh_acti-1554105833651-1-0-SUCCEEDED-root.users.impala-1554105825680.jhist":impala:supergroup:-rwxrwx---

    hdfs中supergroup配置如下:

    dfs.permissions.supergroup=supergroup
    dfs.permissions.superusergroup=supergroup

    解决方法:

    # groupadd supergroup
    # usermod -a -G supergroup mapred
    # usermod -a -G supergroup hdfs
    # usermod -a -G supergroup yarn

    参考:https://community.hortonworks.com/questions/23914/requested-user-hive-is-not-whitelisted-and-has-id.html

  • 相关阅读:
    工业和信息化部“十二五”规划教材:阵列信号处理及MATLAB实现
    PHP 7从零基础到项目实战
    HTML+CSS网页设计详解
    新编 中文版Maya入门与提高
    Lua游戏AI开发指南
    这么设计能热卖:融入运营思维的电商设计进阶宝典(套装上下册)
    WCF初探-1:认识WCF
    用微信二维码登录自己的网站
    01-C语言基本知识
    「OC」内存管理
  • 原文地址:https://www.cnblogs.com/barneywill/p/10628108.html
Copyright © 2011-2022 走看看