zoukankan      html  css  js  c++  java
  • CDH版 hadoop集群启动报错: java.io.IOException: Incompatible clusterIDs

    java.io.IOException: Incompatible clusterIDs in /export/hadoop-2.7.5/hadoopDatas/datanodeDatas2: namenode clusterID = CID-b3356ee2-aaae-4b89-86e2-e6eec8fe6e00; datanode clusterID = CID-c648893d-0a5b-4dc3-ae95-e962e62e0c6c
    at org.apache.hadoop.hdfs.server.datanode.DataStorage.doTransition(DataStorage.java:775)
    at org.apache.hadoop.hdfs.server.datanode.DataStorage.loadStorageDirectory(DataStorage.java:300)
    at org.apache.hadoop.hdfs.server.datanode.DataStorage.loadDataStorage(DataStorage.java:416)
    at org.apache.hadoop.hdfs.server.datanode.DataStorage.addStorageLocations(DataStorage.java:395)
    at org.apache.hadoop.hdfs.server.datanode.DataStorage.recoverTransitionRead(DataStorage.java:573)
    at org.apache.hadoop.hdfs.server.datanode.DataNode.initStorage(DataNode.java:1393)
    at org.apache.hadoop.hdfs.server.datanode.DataNode.initBlockPool(DataNode.java:1358)
    at org.apache.hadoop.hdfs.server.datanode.BPOfferService.verifyAndSetNamespaceInfo(BPOfferService.java:313)
    at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.connectToNNAndHandshake(BPServiceActor.java:216)
    at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:637)
    at java.lang.Thread.run(Thread.java:748)
    2019-11-04 08:51:21,422 FATAL org.apache.hadoop.hdfs.server.datanode.DataNode: Initialization failed for Block pool <registering> (Datanode Uuid unassigned) service to node01/192.168.100.100:8020. Exiting.
    java.io.IOException: All specified directories are failed to load.
    at org.apache.hadoop.hdfs.server.datanode.DataStorage.recoverTransitionRead(DataStorage.java:574)
    at org.apache.hadoop.hdfs.server.datanode.DataNode.initStorage(DataNode.java:1393)
    at org.apache.hadoop.hdfs.server.datanode.DataNode.initBlockPool(DataNode.java:1358)
    at org.apache.hadoop.hdfs.server.datanode.BPOfferService.verifyAndSetNamespaceInfo(BPOfferService.java:313)
    at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.connectToNNAndHandshake(BPServiceActor.java:216)
    at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:637)
    at java.lang.Thread.run(Thread.java:748)

    原因:

    Hadoop启动后,在使用格式化hdfs namenode -format,会导致datanode和namenode的clusterID不一致

    解决办法:

    hdfs-site.xml配置文件中 

    1.namenode上, ${dfs.namenode.name.dir}/current/VERSION 里找到clusterID

    2. 在出问题的datanode上, ${dfs.namenode.data.dir}/current/VERSION 里找到clusterID, 用步骤1中得到的clusterID覆盖之。

    3.重启namenode

  • 相关阅读:
    python3爬取咪咕音乐榜信息(附源代码)
    python3通过Beautif和XPath分别爬取“小猪短租-北京”租房信息,并对比时间效率(附源代码)
    在windows7 32ibt安装MongoDB数据库的方法及连接失败解决方案
    GoWeb开发_Iris框架讲解(一)
    Web项目开发介绍及实战项目介绍
    DAY7学习笔记
    DAY6课下作业
    DAY6学习笔记
    DAY5课下作业
    DAY5学习笔记
  • 原文地址:https://www.cnblogs.com/chong-zuo3322/p/11790523.html
Copyright © 2011-2022 走看看