zoukankan      html  css  js  c++  java
  • hadoop之 node manager起不来, 执行mapreduce 程序hang住

    现象: node manager起不来, 执行mapreduce 程序hang住

    namenode 进程状态查询
    [root@hadp-master sbin]# jps
    8608 ResourceManager
    8257 NameNode
    9268 Jps
    8453 SecondaryNameNode

    datanode 进程状态查询 , 发现 nodemanager 没有起来
    [root@hadp-node1 hadoop]# jps
    4179 DataNode
    4940 Jps

    namenode 单独起 nodemanager
    [root@hadp-master sbin]# yarn nodemanager
    18/01/26 03:53:54 INFO webapp.WebApps: Web app node started at 8042
    18/01/26 03:53:54 INFO client.RMProxy: Connecting to ResourceManager at hadp-master/10.100.25.30:8031
    18/01/26 03:53:54 INFO nodemanager.NodeStatusUpdaterImpl: Sending out 0 NM container statuses: []
    18/01/26 03:53:54 INFO nodemanager.NodeStatusUpdaterImpl: Registering with RM using containers :[]
    18/01/26 03:53:54 ERROR nodemanager.NodeStatusUpdaterImpl: Unexpected error starting NodeStatusUpdater
    org.apache.hadoop.yarn.exceptions.YarnRuntimeException: Recieved SHUTDOWN signal from Resourcemanager ,Registration of NodeManager failed, Message from ResourceManager: NodeManager from hadp-master doesn't satisfy minimum allocations, Sending SHUTDOWN signal to the NodeManager. ## 提示 NodeManager 内存不足


    解决过程:

    1、原虚拟机755m内存,调大虚拟机内存至2G

    2、修改 yarn-site.xml ,添加如下参数

    <property>
    <name>yarn.scheduler.minimum-allocation-mb</name>
    <value>200</value>
    </property>

    <property>
    <name>yarn.scheduler.maximum-allocation-mb</name>
    <value>1600</value>
    </property>

    <property>
    <name>yarn.nodemanager.resource.memory-mb</name>
    <value>1600</value>
    </property>

    <property>
    <name>yarn.nodemanager.resource.cpu-vcores</name>
    <value>1</value>
    </property>


    参数 : yarn.scheduler.minimum-allocation-mb
    默认 : 1024 m
    官方解释: The minimum allocation for every container request at the RM, in MBs. Memory requests lower than this will throw a InvalidResourceRequestException.

    参数 : yarn.scheduler.maximum-allocation-mb
    默认 : 8192 m
    官方解释: The maximum allocation for every container request at the RM, in MBs. Memory requests higher than this will throw a InvalidResourceRequestException.

    参数 : yarn.nodemanager.resource.memory-mb
    默认 : 8192 m
    官方解释: Amount of physical memory, in MB, that can be allocated for containers.

    参数 : yarn.nodemanager.resource.cpu-vcores
    默认 : 8
    官方解释: Number of vcores that can be allocated for containers. This is used by the RM scheduler when allocating resources for containers. This is not used to limit the number of physical cores used by YARN containers.

  • 相关阅读:
    使用模拟器混淆前端代码
    中间人攻击 -- Cookie 喷发
    【探索】机器指令翻译成 JavaScript
    复杂的 Hash 函数组合有意义吗?
    【探索】利用 canvas 实现数据压缩
    【趣事】用 JavaScript 对抗 DDOS 攻击 (下)
    【趣事】用 JavaScript 对抗 DDOS 攻击
    【探索】在 JavaScript 中使用 C 程序
    【探索】无形验证码 —— PoW 算力验证
    对抗明文口令泄露 —— Web 前端慢 Hash
  • 原文地址:https://www.cnblogs.com/andy6/p/8372184.html
Copyright © 2011-2022 走看看