zoukankan      html  css  js  c++  java
  • Hadoop运行任务时一直卡在: INFO mapreduce.Job: Running job

    可能原因:

    1,配置文件写错了,服务没有完全起来

    master:

    [hadoop@master hadoop]$ jps
    33792 SecondaryNameNode
    34223 Jps
    33601 NameNode
    33943 ResourceManager

    slave:

    [hadoop@slave1 hadoop]$ jps
    47298 NodeManager
    47480 Jps
    47185 DataNode

    我的不是这个entire,然后查看日志:

    [hadoop@slave1 logs]$ vim yarn-hadoop-nodemanager-slave1.log

    发现

    2017-07-13 23:17:39,954 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8031. Already tried 1 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
    2017-07-13 23:17:40,955 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8031. Already tried 2 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
    2017-07-13 23:17:41,957 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8031. Already tried 3 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
    2017-07-13 23:17:42,958 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8031. Already tried 4 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
    2017-07-13 23:17:43,959 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8031. Already tried 5 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
    2017-07-13 23:17:44,961 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8031. Already tried 6 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
    2017-07-13 23:17:45,962 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8031. Already tried 7 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
    2017-07-13 23:17:46,963 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8031. Already tried 8 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
    2017-07-13 23:17:47,964 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8031. Already tried 9 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
    2017-07-13 23:17:47,967 ERROR org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl: Unexpected error starting NodeStatusUpdater
    java.net.ConnectException: Call From slave1/10.3.236.241 to 0.0.0.0:8031 failed on connection exception: java.net.ConnectException: Connection refused; For more details see: http://wiki.apache.org/hadoop/ConnectionRefused

    百度后了解到:

    需要在yarn-site.xml中添加如下代码

    <property>
    <name>yarn.resourcemanager.address</name>
    <value>master:8032</value>
    </property>
    <property>
    <name>yarn.resourcemanager.scheduler.address</name>
    <value>master:8030</value>
    </property>
    <property>
    <name>yarn.resourcemanager.resource-tracker.address</name>
    <value>master:8031</value>
    </property>

    重启服务后

    再运行mapreduce发现可疑运行了

    问题解决

  • 相关阅读:
    通过git命令行从github或服务器上克隆、修改和更新项目
    将项目部署到 github上(部署到码云操作一样,前提是有码云账号)
    tp5中代替tp3.2中的一些方法
    关于ThinkPHP下使用Uploadify插件 仅有火狐提示HTTP Error (302)错误的解决办法
    nginx轮询宕机解决
    nginx实现动静分离
    nginx实现负载均衡
    nginx实现反向代理
    解决表单重复提交
    解决跨域问题
  • 原文地址:https://www.cnblogs.com/cityinthesky/p/7168971.html
Copyright © 2011-2022 走看看