zoukankan      html  css  js  c++  java
  • Hadoop运行任务时一直卡在: INFO mapreduce.Job: Running job

    可能原因:

    1,配置文件写错了,服务没有完全起来

    master:

    [hadoop@master hadoop]$ jps
    33792 SecondaryNameNode
    34223 Jps
    33601 NameNode
    33943 ResourceManager

    slave:

    [hadoop@slave1 hadoop]$ jps
    47298 NodeManager
    47480 Jps
    47185 DataNode

    我的不是这个entire,然后查看日志:

    [hadoop@slave1 logs]$ vim yarn-hadoop-nodemanager-slave1.log

    发现

    2017-07-13 23:17:39,954 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8031. Already tried 1 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
    2017-07-13 23:17:40,955 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8031. Already tried 2 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
    2017-07-13 23:17:41,957 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8031. Already tried 3 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
    2017-07-13 23:17:42,958 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8031. Already tried 4 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
    2017-07-13 23:17:43,959 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8031. Already tried 5 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
    2017-07-13 23:17:44,961 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8031. Already tried 6 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
    2017-07-13 23:17:45,962 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8031. Already tried 7 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
    2017-07-13 23:17:46,963 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8031. Already tried 8 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
    2017-07-13 23:17:47,964 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8031. Already tried 9 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
    2017-07-13 23:17:47,967 ERROR org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl: Unexpected error starting NodeStatusUpdater
    java.net.ConnectException: Call From slave1/10.3.236.241 to 0.0.0.0:8031 failed on connection exception: java.net.ConnectException: Connection refused; For more details see: http://wiki.apache.org/hadoop/ConnectionRefused

    百度后了解到:

    需要在yarn-site.xml中添加如下代码

    <property>
    <name>yarn.resourcemanager.address</name>
    <value>master:8032</value>
    </property>
    <property>
    <name>yarn.resourcemanager.scheduler.address</name>
    <value>master:8030</value>
    </property>
    <property>
    <name>yarn.resourcemanager.resource-tracker.address</name>
    <value>master:8031</value>
    </property>

    重启服务后

    再运行mapreduce发现可疑运行了

    问题解决

  • 相关阅读:
    Spring学习记录-源码构建
    Spring学习记录
    java相关技术提纲
    Seata 分布式事务框架
    SourceTree学习记录
    其他源码管理工具
    分布式事务
    Score (模拟题)
    大整数排序
    字符串的查找删除 时间限制:1 Seconds 内存限制:32 Mbyte
  • 原文地址:https://www.cnblogs.com/cityinthesky/p/7168971.html
Copyright © 2011-2022 走看看