起初是验证 On-Yarn模式提交Spark任务时发现连接 namenode:9000 失败,
进一步,在namenode节点机执行 start-dfs.sh后,直接telnet namenode 9000 返回连接拒绝,
再次排查问题,发现NameNode进程启动后立刻挂了,然后就有如下的解决操作:
1、在namenode节点机单启NameNode进程
hadoop-daemon.sh start namenode
2、立即jps两次,发现NameNode进程挂掉
3、查看日志
tail -200 hadoop-hadoop-namenode-storm-01.log
2020-06-16 10:46:43,233 WARN org.apache.hadoop.hdfs.server.common.Storage: Storage directory /home/hadoop/dataDir/hadoop/dfs/name does not exist
2020-06-16 10:46:43,234 WARN org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Encountered exception loading fsimage
org.apache.hadoop.hdfs.server.common.InconsistentFSStateException: Directory /home/hadoop/dataDir/hadoop/dfs/name is in an inconsistent state: storage directory does not exist or is not accessible.
4、新建了目录/home/hadoop/dataDir/hadoop/dfs/name,再次启动namenode,还是报错:
2020-06-16 11:03:11,705 INFO org.apache.hadoop.hdfs.server.common.Storage: Lock on /home/hadoop/dataDir/hadoop/dfs/name/in_use.lock acquired by nodename 5821@storm-01
2020-06-16 11:03:11,771 WARN org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Encountered exception loading fsimage
java.io.IOException: NameNode is not formatted.
5、重新format
删除三台机器上的相关目录
rm -rf /home/hadoop/dataDir/hadoop/*
hdfs namenode -format