zoukankan      html  css  js  c++  java
  • CDH版 hadoop集群启动报错: java.io.IOException: Incompatible clusterIDs

    java.io.IOException: Incompatible clusterIDs in /export/hadoop-2.7.5/hadoopDatas/datanodeDatas2: namenode clusterID = CID-b3356ee2-aaae-4b89-86e2-e6eec8fe6e00; datanode clusterID = CID-c648893d-0a5b-4dc3-ae95-e962e62e0c6c
    at org.apache.hadoop.hdfs.server.datanode.DataStorage.doTransition(DataStorage.java:775)
    at org.apache.hadoop.hdfs.server.datanode.DataStorage.loadStorageDirectory(DataStorage.java:300)
    at org.apache.hadoop.hdfs.server.datanode.DataStorage.loadDataStorage(DataStorage.java:416)
    at org.apache.hadoop.hdfs.server.datanode.DataStorage.addStorageLocations(DataStorage.java:395)
    at org.apache.hadoop.hdfs.server.datanode.DataStorage.recoverTransitionRead(DataStorage.java:573)
    at org.apache.hadoop.hdfs.server.datanode.DataNode.initStorage(DataNode.java:1393)
    at org.apache.hadoop.hdfs.server.datanode.DataNode.initBlockPool(DataNode.java:1358)
    at org.apache.hadoop.hdfs.server.datanode.BPOfferService.verifyAndSetNamespaceInfo(BPOfferService.java:313)
    at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.connectToNNAndHandshake(BPServiceActor.java:216)
    at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:637)
    at java.lang.Thread.run(Thread.java:748)
    2019-11-04 08:51:21,422 FATAL org.apache.hadoop.hdfs.server.datanode.DataNode: Initialization failed for Block pool <registering> (Datanode Uuid unassigned) service to node01/192.168.100.100:8020. Exiting.
    java.io.IOException: All specified directories are failed to load.
    at org.apache.hadoop.hdfs.server.datanode.DataStorage.recoverTransitionRead(DataStorage.java:574)
    at org.apache.hadoop.hdfs.server.datanode.DataNode.initStorage(DataNode.java:1393)
    at org.apache.hadoop.hdfs.server.datanode.DataNode.initBlockPool(DataNode.java:1358)
    at org.apache.hadoop.hdfs.server.datanode.BPOfferService.verifyAndSetNamespaceInfo(BPOfferService.java:313)
    at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.connectToNNAndHandshake(BPServiceActor.java:216)
    at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:637)
    at java.lang.Thread.run(Thread.java:748)

    原因:

    Hadoop启动后,在使用格式化hdfs namenode -format,会导致datanode和namenode的clusterID不一致

    解决办法:

    hdfs-site.xml配置文件中 

    1.namenode上, ${dfs.namenode.name.dir}/current/VERSION 里找到clusterID

    2. 在出问题的datanode上, ${dfs.namenode.data.dir}/current/VERSION 里找到clusterID, 用步骤1中得到的clusterID覆盖之。

    3.重启namenode

  • 相关阅读:
    数据库访问性能优化(转)
    Mysql分表和分区的区别、分库分表介绍与区别
    怎样玩转千万级别的数据(表分区)
    关于Blocking IO,non-Blokcing IO,async IO的区别和理解
    spring security四种实现方式
    使用百度网盘+Git,把版本控制托管到云端,附精彩评论
    Linux下C/C++帮助手册安装方法
    GNU自动化工具使用全过程详解,以及在线手册
    autotools工具使用 good
    使用 GNU Libtool 创建库
  • 原文地址:https://www.cnblogs.com/chong-zuo3322/p/11790523.html
Copyright © 2011-2022 走看看