Apache Hadoop 2.9.2 的YARN High Available 模式部署
作者：尹正杰
版权声明：原创作品，谢绝转载！否则将追究法律责任。
一.环境准备
1>.官方文档（http://hadoop.apache.org/docs/r2.9.2/hadoop-yarn/hadoop-yarn-site/ResourceManagerHA.html）
2>.实验环境
环境在上一篇博客基础之上，参考链接：https://www.cnblogs.com/yinzhengjie/p/10720586.html。
二.修改核心配置
1>.修改yarn-site.xml
[root@node101.yinzhengjie.org.cn ~]# cat /yinzhengjie/softwares/hadoop-2.9.2/etc/hadoop/yarn-site.xml <configuration> <property> <name>yarn.nodemanager.aux-services</name> <value>mapreduce_shuffle</value> </property> <!--启用resourcemanager ha--> <property> <name>yarn.resourcemanager.ha.enabled</name> <value>true</value> </property> <!--声明两台resourcemanager的地址--> <property> <name>yarn.resourcemanager.cluster-id</name> <value>yinzhengjie-yarn</value> </property> <property> <name>yarn.resourcemanager.ha.rm-ids</name> <value>resourcemanager1,resourcemanager2</value> </property> <property> <name>yarn.resourcemanager.hostname.resourcemanager1</name> <value>node101.yinzhengjie.org.cn</value> </property> <property> <name>yarn.resourcemanager.hostname.resourcemanager2</name> <value>node103.yinzhengjie.org.cn</value> </property> <property> <name>yarn.resourcemanager.webapp.address.resourcemanager1</name> <value>node101.yinzhengjie.org.cn:8088</value> </property> <property> <name>yarn.resourcemanager.webapp.address.resourcemanager2</name> <value>node103.yinzhengjie.org.cn:8088</value> </property> <!--指定zookeeper集群的地址--> <property> <name>yarn.resourcemanager.zk-address</name> <value>node101.yinzhengjie.org.cn:2181,node102.yinzhengjie.org.cn:2181,node101.yinzhengjie.org.cn:2181</value> </property> <!--启用自动恢复--> <property> <name>yarn.resourcemanager.recovery.enabled</name> <value>true</value> </property> <!--指定resourcemanager的状态信息存储在zookeeper集群--> <property> <name>yarn.resourcemanager.store.class</name> <value>org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore</value> </property> </configuration> [root@node101.yinzhengjie.org.cn ~]# [root@node101.yinzhengjie.org.cn ~]#
2>.将修改的配置文件同步到其他节点中
[root@node101.yinzhengjie.org.cn ~]# scp /yinzhengjie/softwares/hadoop-2.9.2/etc/hadoop/yarn-site.xml node102.yinzhengjie.org.cn:/yinzhengjie/softwares/hadoop-2.9.2/etc/hadoop/ yarn-site.xml 100% 1566 2.8MB/s 00:00 [root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]# scp /yinzhengjie/softwares/hadoop-2.9.2/etc/hadoop/yarn-site.xml node103.yinzhengjie.org.cn:/yinzhengjie/softwares/hadoop-2.9.2/etc/hadoop/ yarn-site.xml 100% 1566 3.0MB/s 00:00 [root@node101.yinzhengjie.org.cn ~]#
3>.启动hdfs集群
[root@node101.yinzhengjie.org.cn ~]# start-dfs.sh Starting namenodes on [node101.yinzhengjie.org.cn node103.yinzhengjie.org.cn] node101.yinzhengjie.org.cn: starting namenode, logging to /yinzhengjie/softwares/hadoop-2.9.2/logs/hadoop-root-namenode-node101.yinzhengjie.org.cn.out node103.yinzhengjie.org.cn: starting namenode, logging to /yinzhengjie/softwares/hadoop-2.9.2/logs/hadoop-root-namenode-node103.yinzhengjie.org.cn.out node103.yinzhengjie.org.cn: starting datanode, logging to /yinzhengjie/softwares/hadoop-2.9.2/logs/hadoop-root-datanode-node103.yinzhengjie.org.cn.out node102.yinzhengjie.org.cn: starting datanode, logging to /yinzhengjie/softwares/hadoop-2.9.2/logs/hadoop-root-datanode-node102.yinzhengjie.org.cn.out node101.yinzhengjie.org.cn: starting datanode, logging to /yinzhengjie/softwares/hadoop-2.9.2/logs/hadoop-root-datanode-node101.yinzhengjie.org.cn.out Starting journal nodes [node103.yinzhengjie.org.cn node102.yinzhengjie.org.cn node101.yinzhengjie.org.cn] node103.yinzhengjie.org.cn: starting journalnode, logging to /yinzhengjie/softwares/hadoop-2.9.2/logs/hadoop-root-journalnode-node103.yinzhengjie.org.cn.out node102.yinzhengjie.org.cn: starting journalnode, logging to /yinzhengjie/softwares/hadoop-2.9.2/logs/hadoop-root-journalnode-node102.yinzhengjie.org.cn.out node101.yinzhengjie.org.cn: starting journalnode, logging to /yinzhengjie/softwares/hadoop-2.9.2/logs/hadoop-root-journalnode-node101.yinzhengjie.org.cn.out Starting ZK Failover Controllers on NN hosts [node101.yinzhengjie.org.cn node103.yinzhengjie.org.cn] node101.yinzhengjie.org.cn: starting zkfc, logging to /yinzhengjie/softwares/hadoop-2.9.2/logs/hadoop-root-zkfc-node101.yinzhengjie.org.cn.out node103.yinzhengjie.org.cn: starting zkfc, logging to /yinzhengjie/softwares/hadoop-2.9.2/logs/hadoop-root-zkfc-node103.yinzhengjie.org.cn.out [root@node101.yinzhengjie.org.cn ~]# [root@node101.yinzhengjie.org.cn ~]# hdfs haadmin -getServiceState namenode1 #查看当前节点的状态！很显然，和下面的Web UI展示的结果一致！ active [root@node101.yinzhengjie.org.cn ~]#
4>.启动yarn集群
[root@node101.yinzhengjie.org.cn ~]# start-yarn.sh starting yarn daemons starting resourcemanager, logging to /yinzhengjie/softwares/hadoop-2.9.2/logs/yarn-root-resourcemanager-node101.yinzhengjie.org.cn.out node102.yinzhengjie.org.cn: starting nodemanager, logging to /yinzhengjie/softwares/hadoop-2.9.2/logs/yarn-root-nodemanager-node102.yinzhengjie.org.cn.out node103.yinzhengjie.org.cn: starting nodemanager, logging to /yinzhengjie/softwares/hadoop-2.9.2/logs/yarn-root-nodemanager-node103.yinzhengjie.org.cn.out node101.yinzhengjie.org.cn: starting nodemanager, logging to /yinzhengjie/softwares/hadoop-2.9.2/logs/yarn-root-nodemanager-node101.yinzhengjie.org.cn.out [root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]# jps 7024 ZooKeeperMain 2273 Jps 499 QuorumPeerMain 1414 DFSZKFailoverController 1192 JournalNode 824 NameNode 1994 NodeManager 1867 ResourceManager 957 DataNode [root@node101.yinzhengjie.org.cn ~]#
[root@node103.yinzhengjie.org.cn ~]# yarn-daemon.sh start resourcemanager starting resourcemanager, logging to /yinzhengjie/softwares/hadoop-2.9.2/logs/yarn-root-resourcemanager-node103.yinzhengjie.org.cn.out [root@node103.yinzhengjie.org.cn ~]# [root@node103.yinzhengjie.org.cn ~]# jps 7905 Jps 7842 ResourceManager 6984 JournalNode 7545 NodeManager 6554 QuorumPeerMain 6764 NameNode 7132 DFSZKFailoverController 6863 DataNode [root@node103.yinzhengjie.org.cn ~]#
[root@node103.yinzhengjie.org.cn ~]# jps 7842 ResourceManager 6984 JournalNode 7545 NodeManager 6554 QuorumPeerMain 6764 NameNode 7132 DFSZKFailoverController 8653 Jps 6863 DataNode [root@node103.yinzhengjie.org.cn ~]#
[root@node103.yinzhengjie.org.cn ~]# yarn rmadmin -getServiceState resourcemanager1 #很显然，我们先启动的是resourcemanager1节点，因此它优先被选为Active状态！ active [root@node103.yinzhengjie.org.cn ~]# [root@node103.yinzhengjie.org.cn ~]# yarn rmadmin -getServiceState resourcemanager2 standby [root@node103.yinzhengjie.org.cn ~]#
三.验证YARN的高可用性
1>.将resourcemanager1节点进程杀掉，查看服务是否可以正常提供服务
[root@node101.yinzhengjie.org.cn ~]# jps 7024 ZooKeeperMain 499 QuorumPeerMain 1414 DFSZKFailoverController 1192 JournalNode 824 NameNode 1994 NodeManager 1867 ResourceManager 957 DataNode 4191 Jps [root@node101.yinzhengjie.org.cn ~]# [root@node101.yinzhengjie.org.cn ~]# kill -9 `jps | grep ResourceManager | awk '{print $1}' ` [root@node101.yinzhengjie.org.cn ~]# [root@node101.yinzhengjie.org.cn ~]# jps 7024 ZooKeeperMain 499 QuorumPeerMain 4276 Jps 1414 DFSZKFailoverController 1192 JournalNode 824 NameNode 1994 NodeManager 957 DataNode [root@node101.yinzhengjie.org.cn ~]#
2>.启动yarn集群后，注意观察zookeeper集群中生成的目录结构的变化哟！
[root@node101.yinzhengjie.org.cn ~]# zkCli.sh -server node103.yinzhengjie.org.cn:2181 Connecting to node103.yinzhengjie.org.cn:2181 2019-04-19 10:22:52,906 [myid:] - INFO [main:Environment@100] - Client environment:zookeeper.version=3.4.14-4c25d480e66aadd371de8bd2fd8da255ac140bcf, built on 03/06/2019 16:18 GMT 2019-04-19 10:22:52,909 [myid:] - INFO [main:Environment@100] - Client environment:host.name=node101.yinzhengjie.org.cn 2019-04-19 10:22:52,909 [myid:] - INFO [main:Environment@100] - Client environment:java.version=1.8.0_201 2019-04-19 10:22:52,911 [myid:] - INFO [main:Environment@100] - Client environment:java.vendor=Oracle Corporation 2019-04-19 10:22:52,911 [myid:] - INFO [main:Environment@100] - Client environment:java.home=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.201.b09-2.el7_6.x86_64/jre 2019-04-19 10:22:52,911 [myid:] - INFO [main:Environment@100] - Client environment:java.class.path=/yinzhengjie/softwares/zookeeper-3.4.14/bin/../zookeeper-server/target/classes:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../build/classes:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../zookeeper-server/target/lib/*.jar:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../build/lib/*.jar:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../lib/slf4j-log4j12-1.7.25.jar:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../lib/slf4j-api-1.7.25.jar:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../lib/netty-3.10.6.Final.jar:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../lib/log4j-1.2.17.jar:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../lib/jline-0.9.94.jar:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../lib/audience-annotations-0.5.0.jar:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../zookeeper-3.4.14.jar:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../zookeeper-server/src/main/resources/lib/*.jar:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../conf: 2019-04-19 10:22:52,912 [myid:] - INFO [main:Environment@100] - Client environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib 2019-04-19 10:22:52,912 [myid:] - INFO [main:Environment@100] - Client environment:java.io.tmpdir=/tmp 2019-04-19 10:22:52,912 [myid:] - INFO [main:Environment@100] - Client environment:java.compiler=<NA> 2019-04-19 10:22:52,912 [myid:] - INFO [main:Environment@100] - Client environment:os.name=Linux 2019-04-19 10:22:52,912 [myid:] - INFO [main:Environment@100] - Client environment:os.arch=amd64 2019-04-19 10:22:52,912 [myid:] - INFO [main:Environment@100] - Client environment:os.version=3.10.0-957.el7.x86_64 2019-04-19 10:22:52,912 [myid:] - INFO [main:Environment@100] - Client environment:user.name=root 2019-04-19 10:22:52,912 [myid:] - INFO [main:Environment@100] - Client environment:user.home=/root 2019-04-19 10:22:52,912 [myid:] - INFO [main:Environment@100] - Client environment:user.dir=/root 2019-04-19 10:22:52,913 [myid:] - INFO [main:ZooKeeper@442] - Initiating client connection, connectString=node103.yinzhengjie.org.cn:2181 sessionTimeout=30000 watcher=org.apache.zookeeper.ZooKeeperMain$MyWatcher@5ce65a89 Welcome to ZooKeeper! 2019-04-19 10:22:52,938 [myid:] - INFO [main-SendThread(node103.yinzhengjie.org.cn:2181):ClientCnxn$SendThread@1025] - Opening socket connection to server node103.yinzhengjie.org.cn/172.30.1.103:2181. Will not attempt to authenticate using SASL (unknown error) JLine support is enabled 2019-04-19 10:22:52,998 [myid:] - INFO [main-SendThread(node103.yinzhengjie.org.cn:2181):ClientCnxn$SendThread@879] - Socket connection established to node103.yinzhengjie.org.cn/172.30.1.103:2181, initiating session [zk: node103.yinzhengjie.org.cn:2181(CONNECTING) 0] 2019-04-19 10:22:53,026 [myid:] - INFO [main-SendThread(node103.yinzhengjie.org.cn:2181):ClientCnxn$SendThread@1299] - Session establishment complete on server node103.yinzhengjie.org.cn/172.30.1.103:2181, sessionid = 0x6700035fda3a0000, negotiated timeout = 30000 WATCHER:: WatchedEvent state:SyncConnected type:None path:null [zk: node103.yinzhengjie.org.cn:2181(CONNECTED) 0] ls / [zookeeper,hadoop-ha] [zk: node103.yinzhengjie.org.cn:2181(CONNECTED) 1] [zk: node103.yinzhengjie.org.cn:2181(CONNECTED) 1] ls / [zookeeper, yarn-leader-election, hadoop-ha, rmstore] [zk: node103.yinzhengjie.org.cn:2181(CONNECTED) 2]
[zk: node103.yinzhengjie.org.cn:2181(CONNECTED) 2] get /yarn-leader-election cZxid = 0x300000008 ctime = Fri Apr 19 10:09:04 CST 2019 mZxid = 0x300000008 mtime = Fri Apr 19 10:09:04 CST 2019 pZxid = 0x300000009 cversion = 1 dataVersion = 0 aclVersion = 0 ephemeralOwner = 0x0 dataLength = 0 numChildren = 1 [zk: node103.yinzhengjie.org.cn:2181(CONNECTED) 3]
[zk: node103.yinzhengjie.org.cn:2181(CONNECTED) 3] ls /yarn-leader-election [yinzhengjie-yarn] [zk: node103.yinzhengjie.org.cn:2181(CONNECTED) 4]
[zk: node103.yinzhengjie.org.cn:2181(CONNECTED) 4] ls /yarn-leader-election/yinzhengjie-yarn [ActiveBreadCrumb, ActiveStandbyElectorLock] [zk: node103.yinzhengjie.org.cn:2181(CONNECTED) 5]
[zk: node103.yinzhengjie.org.cn:2181(CONNECTED) 7] get /yarn-leader-election/yinzhengjie-yarn/ActiveBreadCrumb yinzhengjie-yarnresourcemanager2 cZxid = 0x30000000b ctime = Fri Apr 19 10:09:08 CST 2019 mZxid = 0x30000002c mtime = Fri Apr 19 10:18:50 CST 2019 pZxid = 0x30000000b cversion = 0 dataVersion = 1 aclVersion = 0 ephemeralOwner = 0x0 dataLength = 36 numChildren = 0 [zk: node103.yinzhengjie.org.cn:2181(CONNECTED) 8]
[zk: node103.yinzhengjie.org.cn:2181(CONNECTED) 8] get /yarn-leader-election/yinzhengjie-yarn/ActiveStandbyElectorLock yinzhengjie-yarnresourcemanager2 cZxid = 0x30000002b ctime = Fri Apr 19 10:18:50 CST 2019 mZxid = 0x30000002b mtime = Fri Apr 19 10:18:50 CST 2019 pZxid = 0x30000002b cversion = 0 dataVersion = 0 aclVersion = 0 ephemeralOwner = 0x660003eeeeb10001 dataLength = 36 numChildren = 0 [zk: node103.yinzhengjie.org.cn:2181(CONNECTED) 9]