Apache Hadoop集群缩容实战案例
作者：尹正杰
版权声明：原创作品，谢绝转载！否则将追究法律责任。
一.黑名单退役
1>.在所有节点未退役之前，查看NameNode的WebUI
2>.编写黑名单的配置文件
[root@hadoop101.yinzhengjie.org.cn ~]# echo ${HADOOP_HOME} /yinzhengjie/softwares/hadoop-2.10.0 [root@hadoop101.yinzhengjie.org.cn ~]# [root@hadoop101.yinzhengjie.org.cn ~]# vim ${HADOOP_HOME}/etc/hadoop/blacklist [root@hadoop101.yinzhengjie.org.cn ~]# [root@hadoop101.yinzhengjie.org.cn ~]# cat ${HADOOP_HOME}/etc/hadoop/blacklist hadoop104.yinzhengjie.org.cn [root@hadoop101.yinzhengjie.org.cn ~]#
3>.进修改NameNode节点的hdfs-site.xml配置文件
[root@hadoop101.yinzhengjie.org.cn ~]# vim ${HADOOP_HOME}/etc/hadoop/hdfs-site.xml
[root@hadoop101.yinzhengjie.org.cn ~]#
[root@hadoop101.yinzhengjie.org.cn ~]# cat ${HADOOP_HOME}/etc/hadoop/hdfs-site.xml
<?xml version="1.0" encoding="UTF-8"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License. See accompanying LICENSE file.
-->
<!-- Put site-specific property overrides in this file. -->
<configuration>
<property>
<name>dfs.replication</name>
<value>3</value>
<description>指定HDFS副本的数量</description>
</property>
<property>
<name>dfs.namenode.secondary.http-address</name>
<value>hadoop105.yinzhengjie.org.cn:50090</value>
<description>指定Hadoop辅助名称节点主机配置</description>
</property>
<property>
<name>dfs.hosts.exclude</name>
<value>/yinzhengjie/softwares/hadoop-2.10.0/etc/hadoop/blacklist</value>
<description>指定黑名单路径，建议大家不要写路径的变量名称，有可能存在无法识别变量的可能性，建议大家直接写黑名单的名称。</description>
</property>
</configuration>
[root@hadoop101.yinzhengjie.org.cn ~]#
4>.刷新NameNode和ResourceMangaer的配置
[root@hadoop101.yinzhengjie.org.cn ~]# hdfs dfsadmin -refreshNodes #其实我测试执行该步骤等待一段时间后并没有生效，于是我重启HDFS集群后发现配置生效啦，但是在Hadoop 7.3版本测试时的确时好使的。
Refresh nodes successful
[root@hadoop101.yinzhengjie.org.cn ~]#
[root@hadoop101.yinzhengjie.org.cn ~]# stop-dfs.sh Stopping namenodes on [hadoop101.yinzhengjie.org.cn] hadoop101.yinzhengjie.org.cn: stopping namenode hadoop102.yinzhengjie.org.cn: stopping datanode hadoop104.yinzhengjie.org.cn: stopping datanode hadoop103.yinzhengjie.org.cn: stopping datanode Stopping secondary namenodes [hadoop105.yinzhengjie.org.cn] hadoop105.yinzhengjie.org.cn: stopping secondarynamenode [root@hadoop101.yinzhengjie.org.cn ~]#
[root@hadoop101.yinzhengjie.org.cn ~]# start-dfs.sh Starting namenodes on [hadoop101.yinzhengjie.org.cn] hadoop101.yinzhengjie.org.cn: starting namenode, logging to /yinzhengjie/softwares/hadoop-2.10.0/logs/hadoop-root-namenode-hadoop101.yinzhengjie.org.cn.out hadoop102.yinzhengjie.org.cn: starting datanode, logging to /yinzhengjie/softwares/hadoop-2.10.0/logs/hadoop-root-datanode-hadoop102.yinzhengjie.org.cn.out hadoop104.yinzhengjie.org.cn: starting datanode, logging to /yinzhengjie/softwares/hadoop-2.10.0/logs/hadoop-root-datanode-hadoop104.yinzhengjie.org.cn.out hadoop103.yinzhengjie.org.cn: starting datanode, logging to /yinzhengjie/softwares/hadoop-2.10.0/logs/hadoop-root-datanode-hadoop103.yinzhengjie.org.cn.out Starting secondary namenodes [hadoop105.yinzhengjie.org.cn] hadoop105.yinzhengjie.org.cn: starting secondarynamenode, logging to /yinzhengjie/softwares/hadoop-2.10.0/logs/hadoop-root-secondarynamenode-hadoop105.yinzhengjie.org.cn.out [root@hadoop101.yinzhengjie.org.cn ~]#
5>.再次查看WebUI
二.使用白名单退役
1>.查看NameNode的WebUI
2>.编辑白名单
[root@hadoop101.yinzhengjie.org.cn ~]# vim ${HADOOP_HOME}/etc/hadoop/whitelist
[root@hadoop101.yinzhengjie.org.cn ~]#
[root@hadoop101.yinzhengjie.org.cn ~]# cat ${HADOOP_HOME}/etc/hadoop/whitelist
hadoop102.yinzhengjie.org.cn
hadoop103.yinzhengjie.org.cn
hadoop107.yinzhengjie.org.cn
[root@hadoop101.yinzhengjie.org.cn ~]#
3>.修改配置文件
[root@hadoop101.yinzhengjie.org.cn ~]# vim ${HADOOP_HOME}/etc/hadoop/hdfs-site.xml
[root@hadoop101.yinzhengjie.org.cn ~]#
[root@hadoop101.yinzhengjie.org.cn ~]# cat ${HADOOP_HOME}/etc/hadoop/hdfs-site.xml
<?xml version="1.0" encoding="UTF-8"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License. See accompanying LICENSE file.
-->
<!-- Put site-specific property overrides in this file. -->
<configuration>
<property>
<name>dfs.replication</name>
<value>3</value>
<description>指定HDFS副本的数量</description>
</property>
<property>
<name>dfs.namenode.secondary.http-address</name>
<value>hadoop105.yinzhengjie.org.cn:50090</value>
<description>指定Hadoop辅助名称节点主机配置</description>
</property>
<property>
<name>dfs.hosts</name>
<value>/yinzhengjie/softwares/hadoop-2.10.0/etc/hadoop/whitelist</value>
<description>指定白名单路径,建议不要使用变量名代替路径，直接写绝对路径</description>
</property>
</configuration>
[root@hadoop101.yinzhengjie.org.cn ~]#
4>. 刷新NameNode的信息(其实此步骤在hadoop 2.10.0版本中并不好使，因此博主认为该步骤可以直接省略，重启集群后配置自然就生效啦)
[root@hadoop101.yinzhengjie.org.cn ~]# hdfs dfsadmin -refreshNodes
Refresh nodes successful
[root@hadoop101.yinzhengjie.org.cn ~]#
5>.重启集群
[root@hadoop101.yinzhengjie.org.cn ~]# stop-dfs.sh Stopping namenodes on [hadoop101.yinzhengjie.org.cn] hadoop101.yinzhengjie.org.cn: stopping namenode hadoop104.yinzhengjie.org.cn: stopping datanode hadoop102.yinzhengjie.org.cn: stopping datanode hadoop103.yinzhengjie.org.cn: stopping datanode Stopping secondary namenodes [hadoop105.yinzhengjie.org.cn] hadoop105.yinzhengjie.org.cn: stopping secondarynamenode [root@hadoop101.yinzhengjie.org.cn ~]#
[root@hadoop101.yinzhengjie.org.cn ~]# start-dfs.sh Starting namenodes on [hadoop101.yinzhengjie.org.cn] hadoop101.yinzhengjie.org.cn: starting namenode, logging to /yinzhengjie/softwares/hadoop-2.10.0/logs/hadoop-root-namenode-hadoop101.yinzhengjie.org.cn.out hadoop102.yinzhengjie.org.cn: starting datanode, logging to /yinzhengjie/softwares/hadoop-2.10.0/logs/hadoop-root-datanode-hadoop102.yinzhengjie.org.cn.out hadoop104.yinzhengjie.org.cn: starting datanode, logging to /yinzhengjie/softwares/hadoop-2.10.0/logs/hadoop-root-datanode-hadoop104.yinzhengjie.org.cn.out hadoop103.yinzhengjie.org.cn: starting datanode, logging to /yinzhengjie/softwares/hadoop-2.10.0/logs/hadoop-root-datanode-hadoop103.yinzhengjie.org.cn.out Starting secondary namenodes [hadoop105.yinzhengjie.org.cn] hadoop105.yinzhengjie.org.cn: starting secondarynamenode, logging to /yinzhengjie/softwares/hadoop-2.10.0/logs/hadoop-root-secondarynamenode-hadoop105.yinzhengjie.org.cn.out [root@hadoop101.yinzhengjie.org.cn ~]#
6>.再次查看NameNode的WebUI
7>.查看hadoop104.yinzhengjie.org.cn节点的日志信息
[root@hadoop104.yinzhengjie.org.cn ~]# tail -17 /yinzhengjie/softwares/hadoop-2.10.0/logs/hadoop-root-datanode-hadoop104.yinzhengjie.org.cn.log 2020-03-17 22:22:28,866 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Block pool BP-1625463285-172.200.4.101-1583947083452 (Datanode Uuid e3618a42-aca6-46d4-95ba-643f037fd8ef) service to hadoop101.yinzhengjie.org.cn/172.200.4.101:9000 beginning handshake with NN 2020-03-17 22:22:28,870 ERROR org.apache.hadoop.hdfs.server.datanode.DataNode: Initialization failed for Block pool BP-1625463285-172.200.4.101-1583947083452 (Datanode Uuid e3618a42-aca6-46d4-95ba-643f037fd8ef) service to hadoop101.yinzhengjie.org.cn/172.200.4.101:9000 Datanode denied communication with namenode because the host is not in the include-list: DatanodeRegistration(172.200.4.104:50010, datanodeUuid=e3618a42-aca6-46d4-95ba-643f037fd8ef, infoPort=50075, infoSecurePort=0, ipcPort=50020, storageInfo=lv=-57;cid=CID-394a06a1-4f7f-4430-92e6-3d9414c25669;nsid=191653871;c=1583947083452) at org.apache.hadoop.hdfs.server.blockmanagement.DatanodeManager.registerDatanode(DatanodeManager.java:965) at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.registerDatanode(BlockManager.java:2049) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.registerDatanode(FSNamesystem.java:3694) at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.registerDatanode(NameNodeRpcServer.java:1466) at org.apache.hadoop.hdfs.protocolPB.DatanodeProtocolServerSideTranslatorPB.registerDatanode(DatanodeProtocolServerSideTranslatorPB.java:101) at org.apache.hadoop.hdfs.protocol.proto.DatanodeProtocolProtos$DatanodeProtocolService$2.callBlockingMethod(DatanodeProtocolProtos.java:30583) at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:507) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1034) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:994) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:922) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1893) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2833) [root@hadoop104.yinzhengjie.org.cn ~]# [root@hadoop104.yinzhengjie.org.cn ~]#
8>.温馨提示
很显然，白名单要比黑名单更严谨。
白名单比较霸道，就算DataNode进程存在，DataNode会频繁上报块信息给NameNode，而NameNode始终不会同意不在白名单的DataNode注册到该NameNode管理的HDFS集群。
从WebUI就可以深刻体会到黑名单比白名单更加友善，因为黑名单会显示退役，而使用白名单压根就不会显示已经退役的节点哟，因此，生成环境中建议大家使用黑名单。