Hapoop 搭建 （十二）Hadoop集群启动 （zookeeper、HDFS、YARN、Hbase、Spark）

 

1、启动zookeeper集群  3个节点

sh /opt/modules/zookeeper-3.4.14/bin/zkServer.sh start

[hadoop@centos01 ~]$ sh /opt/modules/zookeeper-3.4.14/bin/zkServer.sh start
ZooKeeper JMX enabled by default
Using config: /opt/modules/zookeeper-3.4.14/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED

查看状态
sh /opt/modules/zookeeper-3.4.14/bin/zkServer.sh status

[hadoop@centos03 dataDir]$ sh /opt/modules/zookeeper-3.4.14/bin/zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /opt/modules/zookeeper-3.4.14/bin/../conf/zoo.cfg
Mode: leader

2、启动HDFS centos01节点

sh /opt/modules/hadoop-2.8.2/sbin/start-dfs.sh

[hadoop@centos01 ~]$ sh /opt/modules/hadoop-2.8.2/sbin/start-dfs.sh
Starting namenodes on [centos01 centos02]
centos01: starting namenode, logging to /opt/modules/hadoop-2.8.2/logs/hadoop-hadoop-namenode-centos01.out
centos02: starting namenode, logging to /opt/modules/hadoop-2.8.2/logs/hadoop-hadoop-namenode-centos02.out
centos03: starting datanode, logging to /opt/modules/hadoop-2.8.2/logs/hadoop-hadoop-datanode-centos03.out
centos01: starting datanode, logging to /opt/modules/hadoop-2.8.2/logs/hadoop-hadoop-datanode-centos01.out
centos02: starting datanode, logging to /opt/modules/hadoop-2.8.2/logs/hadoop-hadoop-datanode-centos02.out
Starting journal nodes [centos01 centos02 centos03]
centos03: starting journalnode, logging to /opt/modules/hadoop-2.8.2/logs/hadoop-hadoop-journalnode-centos03.out
centos02: starting journalnode, logging to /opt/modules/hadoop-2.8.2/logs/hadoop-hadoop-journalnode-centos02.out
centos01: starting journalnode, logging to /opt/modules/hadoop-2.8.2/logs/hadoop-hadoop-journalnode-centos01.out

[hadoop@centos01 dataDir]$ jps
34278 NameNode
34535 DataNode
32328 QuorumPeerMain
35420 JournalNode
35981 Jps

[hadoop@centos02 dataDir]$ jps
34052 DataNode
31638 QuorumPeerMain
35646 Jps
33855 NameNode
34927 JournalNode


[hadoop@centos03 dataDir]$ jps
29878 QuorumPeerMain
33452 DataNode
35213 Jps
34350 JournalNode

3、启动ZKFC守护进程  centos01、centos02

sh /opt/modules/hadoop-2.8.2/sbin/hadoop-daemon.sh start zkfc

[hadoop@centos01 ~]$ sh /opt/modules/hadoop-2.8.2/sbin/hadoop-daemon.sh start zkfc
starting zkfc, logging to /opt/modules/hadoop-2.8.2/logs/hadoop-hadoop-zkfc-centos01.out

4、验证  http://192.168.0.171:50070   http://192.168.0.172:50070

 5、启动YARN集群 启动ResourceManager进程。  centos01、centos02

sh /opt/modules/hadoop-2.8.2/sbin/yarn-daemon.sh start resourcemanager

[hadoop@centos01 ~]$ sh /opt/modules/hadoop-2.8.2/sbin/yarn-daemon.sh start resourcemanager
starting resourcemanager, logging to /opt/modules/hadoop-2.8.2/logs/yarn-hadoop-resourcemanager-centos01.out

6、启动 NodeManager  3个节点

sh /opt/modules/hadoop-2.8.2/sbin/yarn-daemon.sh start nodemanager

[hadoop@centos01 ~]$ sh /opt/modules/hadoop-2.8.2/sbin/yarn-daemon.sh start nodemanager
starting nodemanager, logging to /opt/modules/hadoop-2.8.2/logs/yarn-hadoop-nodemanager-centos01.out

7、验证 http://192.168.0.171:8088/cluster    http://192.168.0.172:8088/cluster  一个进入，另一个跳转（未配置主机IP映射）

8、查看jps

[hadoop@centos01 sbin]$ jps
ResourceManager
JournalNode
NodeManager
DataNode
QuorumPeerMain
DFSZKFailoverController
Jps
NameNode

[hadoop@centos02 ~]$ jps
DataNode
NodeManager
ResourceManager
Jps
DFSZKFailoverController
QuorumPeerMain
JournalNode
NameNode

[hadoop@centos03 hadoop]$ jps
NodeManager
JournalNode
Jps
QuorumPeerMain
DataNode

 9、启动Hbase

  在centos01节点启动hbase

启动
sh /opt/modules/hbase-2.1.9/bin/start-hbase.sh
停止
sh /opt/modules/hbase-2.1.9/bin/stop-hbase.sh

查看jps
[hadoop@centos01 bin]$ jps
Jps
NameNode
DataNode
QuorumPeerMain
HRegionServer
DFSZKFailoverController
JournalNode
NodeManager
HMaster
ResourceManager

[hadoop@centos02 hbase-2.1.9]$ jps
ResourceManager
Jps
DataNode
QuorumPeerMain
HRegionServer
DFSZKFailoverController
NodeManager
NameNode
JournalNode

[hadoop@centos03 hbase-2.1.9]$ jps
NodeManager
QuorumPeerMain
HRegionServer
DataNode
JournalNode
Jps

访问URL

http://192.168.0.171:16010

 10、启动Spark

在centos01启动 
sh /opt/modules/spark-3.0.0-preview2-bin-hadoop2.7/sbin/start-all.sh
在centos02启动第二个master
sh /opt/modules/spark-3.0.0-preview2-bin-hadoop2.7/sbin/start-master.sh

[hadoop@centos01 conf]$ jps
DataNode
JournalNode
ResourceManager
QuorumPeerMain
DFSZKFailoverController
HRegionServer
HMaster
Jps
NodeManager
Master

[hadoop@centos02 conf]$ jps
DFSZKFailoverController
ResourceManager
HMaster
HRegionServer
JournalNode
Jps
QuorumPeerMain
NameNode
DataNode
NodeManager
Worker
Master

[hadoop@centos03 ~]$ jps
Jps
QuorumPeerMain
NodeManager
DataNode
HRegionServer
Worker
JournalNode

http://192.168.0.171:8080
http://192.168.0.172:8080

结束