zoukankan      html  css  js  c++  java
  • hadoop-3.0.0 配置中的 yarn.nodemanager.aux-services 项

    在hadoop-3.0.0-alpha4 的配置中,yarn.nodemanager.aux-services项的默认值是“mapreduce.shuffle”,但如果在hadoop-2.2 中继续使用这个值,NodeManager 会启动失败,在logs/yarn-biadmin-nodemanager-hostname.log里会报如下的错:

    java.lang.IllegalArgumentException: The ServiceName: mapreduce.shuffle set in yarn.nodemanager.aux-services is invalid.The valid service name should only contain a-zA-Z0-9_ and can not start with numbers

    at com.google.common.base.Preconditions.checkArgument(Preconditions.java:88)

    at org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices.serviceInit(AuxServices.java:115)

    at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)

    at org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:107)

    at org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl.serviceInit(ContainerManagerImpl.java:303)

    at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)

    at org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:107)

    at org.apache.hadoop.yarn.server.nodemanager.NodeManager.serviceInit(NodeManager.java:406)

    at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)

    at org.apache.hadoop.yarn.server.nodemanager.NodeManager.initAndStartNodeManager(NodeManager.java:748)

    at org.apache.hadoop.yarn.server.nodemanager.NodeManager.main(NodeManager.java:809)

    2017-07-21 03:21:42,946 INFO org.apache.hadoop.yarn.server.nodemanager.NodeManager: SHUTDOWN_MSG: 

    /************************************************************

    SHUTDOWN_MSG: Shutting down NodeManager at slave1/127.0.0.1

    ************************************************************/

    解决方案:

    将yarn.nodemanager.aux-services项的值"mapreduce.shuffle"改为“mapreduce_shuffle”。

    注意:需要在每个节点的yarn-site.xml上改,位置:/usr/hadoop/hadoop-3.0.0-alpha4/etc/hadoop

  • 相关阅读:
    快速幂模板
    部分有关素数的题
    POJ 3624 Charm Bracelet (01背包)
    51Nod 1085 背包问题 (01背包)
    POJ 1789 Truck History (Kruskal 最小生成树)
    HDU 1996 汉诺塔VI
    HDU 2511 汉诺塔X
    HDU 2175 汉诺塔IX (递推)
    HDU 2077 汉诺塔IV (递推)
    HDU 2064 汉诺塔III (递推)
  • 原文地址:https://www.cnblogs.com/code4app/p/9300046.html
Copyright © 2011-2022 走看看