zoukankan      html  css  js  c++  java
  • hadoop集群之HDFS和YARN启动和停止命令

    假如我们只有3台linux虚拟机,主机名分别为hadoop01、hadoop02和hadoop03,在这3台机器上,hadoop集群的部署情况如下:

    hadoop01:1个namenode,1个datanode,1个journalnode,1个zkfc,1个resourcemanager,1个nodemanager;
    
    hadoop02:1个namenode,1个datanode,1个journalnode,1个zkfc,1个resourcemanager,1个nodemanager;
    
    hadoop03:1个datenode,1个journalnode,1个nodemanager;

    下面我们来介绍启动hdfs和yarn的一些命令。

    1.启动hdfs集群(使用hadoop的批量启动脚本)

    /root/apps/hadoop/sbin/start-dfs.sh
     
    [root@hadoop01 ~]# /root/apps/hadoop/sbin/start-dfs.sh 
    Starting namenodes on [hadoop01 hadoop02]
    hadoop01: starting namenode, logging to /root/apps/hadoop/logs/hadoop-root-namenode-hadoop01.out
    hadoop02: starting namenode, logging to /root/apps/hadoop/logs/hadoop-root-namenode-hadoop02.out
    hadoop03: starting datanode, logging to /root/apps/hadoop/logs/hadoop-root-datanode-hadoop03.out
    hadoop02: starting datanode, logging to /root/apps/hadoop/logs/hadoop-root-datanode-hadoop02.out
    hadoop01: starting datanode, logging to /root/apps/hadoop/logs/hadoop-root-datanode-hadoop01.out
    Starting journal nodes [hadoop01 hadoop02 hadoop03]
    hadoop03: starting journalnode, logging to /root/apps/hadoop/logs/hadoop-root-journalnode-hadoop03.out
    hadoop02: starting journalnode, logging to /root/apps/hadoop/logs/hadoop-root-journalnode-hadoop02.out
    hadoop01: starting journalnode, logging to /root/apps/hadoop/logs/hadoop-root-journalnode-hadoop01.out
    Starting ZK Failover Controllers on NN hosts [hadoop01 hadoop02]
    hadoop01: starting zkfc, logging to /root/apps/hadoop/logs/hadoop-root-zkfc-hadoop01.out
    hadoop02: starting zkfc, logging to /root/apps/hadoop/logs/hadoop-root-zkfc-hadoop02.out
    [root@hadoop01 ~]# 
     

    从上面的启动日志可以看出,start-dfs.sh这个启动脚本是通过ssh对多个节点的namenode、datanode、journalnode以及zkfc进程进行批量启动的。

    2.停止hdfs集群(使用hadoop的批量启动脚本)

    /root/apps/hadoop/sbin/stop-dfs.sh 
     
    [root@hadoop01 ~]# /root/apps/hadoop/sbin/stop-dfs.sh 
    Stopping namenodes on [hadoop01 hadoop02]
    hadoop02: stopping namenode
    hadoop01: stopping namenode
    hadoop02: stopping datanode
    hadoop03: stopping datanode
    hadoop01: stopping datanode
    Stopping journal nodes [hadoop01 hadoop02 hadoop03]
    hadoop03: stopping journalnode
    hadoop02: stopping journalnode
    hadoop01: stopping journalnode
    Stopping ZK Failover Controllers on NN hosts [hadoop01 hadoop02]
    hadoop01: stopping zkfc
    hadoop02: stopping zkfc
    [root@hadoop01 ~]# 
     

    3.启动单个进程

    [root@hadoop01 ~]# /root/apps/hadoop/sbin/hadoop-daemon.sh start namenode
    starting namenode, logging to /root/apps/hadoop/logs/hadoop-root-namenode-hadoop01.out
    [root@hadoop02 ~]# /root/apps/hadoop/sbin/hadoop-daemon.sh start namenode
    starting namenode, logging to /root/apps/hadoop/logs/hadoop-root-namenode-hadoop02.out
    [root@hadoop01 ~]# /root/apps/hadoop/sbin/hadoop-daemon.sh start datanode
    starting datanode, logging to /root/apps/hadoop/logs/hadoop-root-datanode-hadoop01.out
    [root@hadoop02 ~]# /root/apps/hadoop/sbin/hadoop-daemon.sh start datanode
    starting datanode, logging to /root/apps/hadoop/logs/hadoop-root-datanode-hadoop02.out
    [root@hadoop03 apps]# /root/apps/hadoop/sbin/hadoop-daemon.sh start datanode
    starting datanode, logging to /root/apps/hadoop/logs/hadoop-root-datanode-hadoop03.out
    [root@hadoop01 ~]# /root/apps/hadoop/sbin/hadoop-daemon.sh start journalnode
    starting journalnode, logging to /root/apps/hadoop/logs/hadoop-root-journalnode-hadoop01.out
    [root@hadoop02 ~]# /root/apps/hadoop/sbin/hadoop-daemon.sh start journalnode
    starting journalnode, logging to /root/apps/hadoop/logs/hadoop-root-journalnode-hadoop02.out
    [root@hadoop03 apps]# /root/apps/hadoop/sbin/hadoop-daemon.sh start journalnode
    starting journalnode, logging to /root/apps/hadoop/logs/hadoop-root-journalnode-hadoop03.out
    [root@hadoop01 ~]# /root/apps/hadoop/sbin/hadoop-daemon.sh start zkfc
    starting zkfc, logging to /root/apps/hadoop/logs/hadoop-root-zkfc-hadoop01.out
    [root@hadoop02 ~]# /root/apps/hadoop/sbin/hadoop-daemon.sh start zkfc
    starting zkfc, logging to /root/apps/hadoop/logs/hadoop-root-zkfc-hadoop02.out

     分别查看启动后3台虚拟机上的进程情况:

     
    [root@hadoop01 ~]# jps
    6695 DataNode
    2002 QuorumPeerMain
    6879 DFSZKFailoverController
    7035 Jps
    6800 JournalNode
    6580 NameNode
    [root@hadoop01 ~]# 
     
     
    [root@hadoop02 ~]# jps
    6360 JournalNode
    6436 DFSZKFailoverController
    2130 QuorumPeerMain
    6541 Jps
    6255 DataNode
    6155 NameNode
    [root@hadoop02 ~]# 
     
    [root@hadoop03 apps]# jps
    5331 Jps
    5103 DataNode
    5204 JournalNode
    2258 QuorumPeerMain
    [root@hadoop03 apps]# 

    3.停止单个进程

     
    [root@hadoop01 ~]# jps
    6695 DataNode
    2002 QuorumPeerMain
    8486 Jps
    6879 DFSZKFailoverController
    6800 JournalNode
    6580 NameNode
    [root@hadoop01 ~]# 
    [root@hadoop01 ~]# 
    [root@hadoop01 ~]# 
    [root@hadoop01 ~]# 
    [root@hadoop01 ~]# /root/apps/hadoop/sbin/hadoop-daemon.sh stop zkfc
    stopping zkfc
    [root@hadoop01 ~]# /root/apps/hadoop/sbin/hadoop-daemon.sh stop journalnode
    stopping journalnode
    [root@hadoop01 ~]# /root/apps/hadoop/sbin/hadoop-daemon.sh stop datanode
    stopping datanode
    [root@hadoop01 ~]# /root/apps/hadoop/sbin/hadoop-daemon.sh stop namenode
    stopping namenode
    [root@hadoop01 ~]# jps
    2002 QuorumPeerMain
    8572 Jps
    [root@hadoop01 ~]# 
     
     
    [root@hadoop02 ~]# jps
    6360 JournalNode
    6436 DFSZKFailoverController
    2130 QuorumPeerMain
    7378 Jps
    6255 DataNode
    6155 NameNode
    [root@hadoop02 ~]# /root/apps/hadoop/sbin/hadoop-daemon.sh stop zkfc
    stopping zkfc
    [root@hadoop02 ~]# /root/apps/hadoop/sbin/hadoop-daemon.sh stop journalnode
    stopping journalnode
    [root@hadoop02 ~]# /root/apps/hadoop/sbin/hadoop-daemon.sh stop datanode
    stopping datanode
    [root@hadoop02 ~]# /root/apps/hadoop/sbin/hadoop-daemon.sh stop namenode
    stopping namenode
    [root@hadoop02 ~]# jps
    7455 Jps
    2130 QuorumPeerMain
    [root@hadoop02 ~]# 
     
     
    [root@hadoop03 apps]# jps
    5103 DataNode
    5204 JournalNode
    5774 Jps
    2258 QuorumPeerMain
    [root@hadoop03 apps]# /root/apps/hadoop/sbin/hadoop-daemon.sh stop journalnode
    stopping journalnode
    [root@hadoop03 apps]# /root/apps/hadoop/sbin/hadoop-daemon.sh stop datanode
    stopping datanode
    [root@hadoop03 apps]# jps
    5818 Jps
    2258 QuorumPeerMain
    [root@hadoop03 apps]# 
     

    3.启动yarn集群(使用hadoop的批量启动脚本)

    /root/apps/hadoop/sbin/start-yarn.sh 
     
    [root@hadoop01 ~]# /root/apps/hadoop/sbin/start-yarn.sh 
    starting yarn daemons
    starting resourcemanager, logging to /root/apps/hadoop/logs/yarn-root-resourcemanager-hadoop01.out
    hadoop03: starting nodemanager, logging to /root/apps/hadoop/logs/yarn-root-nodemanager-hadoop03.out
    hadoop02: starting nodemanager, logging to /root/apps/hadoop/logs/yarn-root-nodemanager-hadoop02.out
    hadoop01: starting nodemanager, logging to /root/apps/hadoop/logs/yarn-root-nodemanager-hadoop01.out
    [root@hadoop01 ~]# 
     

    从上面的启动日志可以看出,start-yarn.sh启动脚本只在本地启动一个ResourceManager进程,而3台机器上的nodemanager都是通过ssh的方式启动的。所以hadoop02机器上的ResourceManager需要我们手动去启动。

    4.启动hadoop02上的ResourceManager进程

    /root/apps/hadoop/sbin/yarn-daemon.sh start resourcemanager

     5.停止yarn

    /root/apps/hadoop/sbin/stop-yarn.sh
     
    [root@hadoop01 ~]# /root/apps/hadoop/sbin/stop-yarn.sh 
    stopping yarn daemons
    stopping resourcemanager
    hadoop01: stopping nodemanager
    hadoop03: stopping nodemanager
    hadoop02: stopping nodemanager
    no proxyserver to stop
    [root@hadoop01 ~]# 
     

     通过上面的停止日志可以看出,stop-yarn.sh脚本只停止了本地的那个ResourceManager进程,所以hadoop02上的那个resourcemanager我们需要单独去停止。

    6.停止hadoop02上的resourcemanager

    /root/apps/hadoop/sbin/yarn-daemon.sh stop resourcemanager

    注意:启动和停止单个hdfs相关的进程使用的是"hadoop-daemon.sh"脚本,而启动和停止yarn使用的是"yarn-daemon.sh"脚本。

  • 相关阅读:
    剑指Offer(Java版)第四十六题:输出所有和为S的连续正数序列。序列内按照从小至大的顺序,序列间按照开始数字从小到大的顺序。
    剑指Offer(Java版)第四十五题:一个整型数组里除了两个数字之外,其他的数字都出现了两次。请写程序找出这两个只出现一次的数字。
    剑指Offer(Java版)第四十四题:输入一棵二叉树,判断该二叉树是否是平衡二叉树。
    剑指Offer(Java版)第四十三题:输入一棵二叉树,求该树的深度。 从根结点到叶结点依次经过的结点 (含根、叶结点)形成树的一条路径, 最长路径的长度为树的深度。
    剑指Offer(Java版)第四十二题:统计一个数字在排序数组中出现的次数。
    剑指Offer(Java版)第四十一题:输入两个链表,找出它们的第一个公共结点。
    剑指Offer(Java版)第四十题:在数组中的两个数字,如果前面一个数字大于后面的数字, 则这两个数字组成一个逆序对。输入一个数组,求出这个数组中的逆序对的总数P。 并将P对1000000007取模的结果输出。 即输出P%1000000007
    剑指Offer(Java版)第三十九题:在一个字符串(0<=字符串长度<=10000,全部由字母组成)中找到第一个只出现一次的字符, 并返回它的位置, 如果没有则返回 -1(需要区分大小写).
    剑指Offer(Java版)第三十八题:把只包含质因子2、3和5的数称作丑数(Ugly Number)。 例如6、8都是丑数,但14不是,因为它包含质因子7。 习惯上我们把1当做是第一个丑数。 求按从小到大的顺序的第N个丑数。
    【微信支付】退款
  • 原文地址:https://www.cnblogs.com/liuys635/p/12377877.html
Copyright © 2011-2022 走看看