zoukankan      html  css  js  c++  java
  • yarn.resourcemanager.ha.id设置

    resourcemanager启动报错,其中一个启动成功,另一个启动报8088端口被成功启动的rm占用

    2016-11-18 17:08:49,478 INFO org.apache.zookeeper.ZooKeeper: Client environment:java.library.path=/usr/lib/hadoop/lib/native
    2016-11-18 17:08:49,478 INFO org.apache.zookeeper.ZooKeeper: Client environment:java.io.tmpdir=/tmp
    2016-11-18 17:08:49,478 INFO org.apache.zookeeper.ZooKeeper: Client environment:java.compiler=<NA>
    2016-11-18 17:08:49,478 INFO org.apache.zookeeper.ZooKeeper: Client environment:os.name=Linux
    2016-11-18 17:08:49,478 INFO org.apache.zookeeper.ZooKeeper: Client environment:os.arch=amd64
    2016-11-18 17:08:49,478 INFO org.apache.zookeeper.ZooKeeper: Client environment:os.version=2.6.32-504.el6.x86_64
    2016-11-18 17:08:49,479 INFO org.apache.zookeeper.ZooKeeper: Client environment:user.name=yarn
    2016-11-18 17:08:49,479 INFO org.apache.zookeeper.ZooKeeper: Client environment:user.home=/var/lib/hadoop-yarn
    2016-11-18 17:08:49,479 INFO org.apache.zookeeper.ZooKeeper: Client environment:user.dir=/usr/lib/hadoop-yarn
    2016-11-18 17:08:49,480 INFO org.apache.zookeeper.ZooKeeper: Initiating client connection, connectString=node01:2181,node02:2181,node03:2181 sessionTimeout=10000 watcher=org.apache.hadoop.ha.ActiveStandbyElector$WatcherWithClientRef@729b9707
    2016-11-18 17:08:49,501 INFO org.apache.zookeeper.ClientCnxn: Opening socket connection to server node02/172.16.145.112:2181. Will not attempt to authenticate using SASL (unknown error)
    2016-11-18 17:08:49,507 INFO org.apache.zookeeper.ClientCnxn: Socket connection established to node02/172.16.145.112:2181, initiating session
    2016-11-18 17:08:49,518 INFO org.apache.zookeeper.ClientCnxn: Session establishment complete on server node02/172.16.145.112:2181, sessionid = 0x2587635b7620006, negotiated timeout = 10000
    2016-11-18 17:08:49,521 INFO org.apache.hadoop.ha.ActiveStandbyElector: Session connected.
    2016-11-18 17:08:49,544 INFO org.apache.hadoop.ha.ActiveStandbyElector: Successfully created /yarn-leader-election/myclusterrm in ZK.
    2016-11-18 17:08:49,549 INFO org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Transitioning to standby state
    2016-11-18 17:08:49,549 INFO org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Transitioned to standby state
    2016-11-18 17:08:49,642 INFO org.mortbay.log: Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog
    2016-11-18 17:08:49,648 INFO org.apache.hadoop.http.HttpRequestLog: Http request log for http.requests.resourcemanager is not defined
    2016-11-18 17:08:49,666 INFO org.apache.hadoop.http.HttpServer2: Added global filter 'safety' (class=org.apache.hadoop.http.HttpServer2$QuotingInputFilter)
    2016-11-18 17:08:49,677 INFO org.apache.hadoop.http.HttpServer2: Added filter RMAuthenticationFilter (class=org.apache.hadoop.yarn.server.security.http.RMAuthenticationFilter) to context cluster
    2016-11-18 17:08:49,677 INFO org.apache.hadoop.http.HttpServer2: Added filter RMAuthenticationFilter (class=org.apache.hadoop.yarn.server.security.http.RMAuthenticationFilter) to context static
    2016-11-18 17:08:49,677 INFO org.apache.hadoop.http.HttpServer2: Added filter RMAuthenticationFilter (class=org.apache.hadoop.yarn.server.security.http.RMAuthenticationFilter) to context logs
    2016-11-18 17:08:49,678 INFO org.apache.hadoop.http.HttpServer2: Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context cluster
    2016-11-18 17:08:49,678 INFO org.apache.hadoop.http.HttpServer2: Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context static
    2016-11-18 17:08:49,678 INFO org.apache.hadoop.http.HttpServer2: Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context logs
    2016-11-18 17:08:49,684 INFO org.apache.hadoop.http.HttpServer2: adding path spec: /cluster/*
    2016-11-18 17:08:49,684 INFO org.apache.hadoop.http.HttpServer2: adding path spec: /ws/*
    2016-11-18 17:08:49,701 INFO org.apache.hadoop.http.HttpServer2: HttpServer.start() threw a non Bind IOException
    java.net.BindException: Port in use: 172.16.145.114:8088
            at org.apache.hadoop.http.HttpServer2.openListeners(HttpServer2.java:890)
            at org.apache.hadoop.http.HttpServer2.start(HttpServer2.java:826)
            at org.apache.hadoop.yarn.webapp.WebApps$Builder.start(WebApps.java:273)
            at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.startWepApp(ResourceManager.java:903)
            at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.serviceStart(ResourceManager.java:1014)
            at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
            at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.main(ResourceManager.java:1145)
    Caused by: java.net.BindException: Cannot assign requested address
            at sun.nio.ch.Net.bind0(Native Method)
            at sun.nio.ch.Net.bind(Net.java:444)
            at sun.nio.ch.Net.bind(Net.java:436)
            at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:214)
            at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74)
            at org.mortbay.jetty.nio.SelectChannelConnector.open(SelectChannelConnector.java:216)
            at org.apache.hadoop.http.HttpServer2.openListeners(HttpServer2.java:885)
            ... 6 more
    

    标红的IP:172.16.145.114为成功启动的resourcemanager,上面的报错信息是启动失败的rm(172.16.145.115)报的错

    错误显示114的8088端口已经在使用,跟我115有毛关系啊。

    检查配置文件发现  两个rm中的yarn-site.xml中的 yarn.resourcemanager.ha.id设置成了同一个rm的主机名

    解决办法:rm1的yarn-site的yarn.resourcemanager.ha.id设置成rm1

                  rm2的yarn-site的yarn.resourcemanager.ha.id设置成rm2

    <!--在node04上配rm1,node05上配rm2,slave节点不用配这个属性-->
    <property>
    <name>yarn.resourcemanager.ha.id</name>
    <value>rm1</value>
    </property>
    

    错误总结:一直以为两个resourcemanager配置文件是一样的,没有注意到这个属性。

    对于这个属性两个rm的yarn-site设置不一样,而且slave中不需要配,其他属性暂时还发现,所有节点保持一致就行了

  • 相关阅读:
    字典的操作用法小总结
    HTTP Headers解析
    RStdio常用快捷键
    R语言数据类型
    数据科学实战手册(R+Python)书中引用资料网址
    ggplot2使用初探
    urllib2使用初探
    R语言以及RStdio的安装
    目标检测--Selective Search for Object Recognition(IJCV, 2013)
    关于Python的lambda
  • 原文地址:https://www.cnblogs.com/zwgblog/p/6079285.html
Copyright © 2011-2022 走看看