zoukankan      html  css  js  c++  java
  • Hadoop问题:启动hadoop 2.6遇到的datanode启动不了

    问题描述:第一次启动输入jps都有,第二次没有datanode

    日志如下:  

    查看日志如下:
    
    2014-12-22 12:08:27,264 INFO org.mortbay.log: Started HttpServer2$SelectChannelConnectorWithSafeStartup@0.0.0.0:50075
    2014-12-22 12:08:27,692 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: dnUserName = root
    2014-12-22 12:08:27,692 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: supergroup = supergroup
    2014-12-22 12:08:32,865 INFO org.apache.hadoop.ipc.CallQueueManager: Using callQueue class java.util.concurrent.LinkedBlockingQueue
    2014-12-22 12:08:32,889 INFO org.apache.hadoop.ipc.Server: Starting Socket Reader #1 for port 50020
    2014-12-22 12:08:32,931 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Opened IPC server at /0.0.0.0:50020
    2014-12-22 12:08:32,945 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Refresh request received for nameservices: null
    2014-12-22 12:08:32,968 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Starting BPOfferServices for nameservices: <default>
    2014-12-22 12:08:32,992 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Block pool <registering> (Datanode Uuid unassigned) service to localhost/127.0.0.1:8020 starting to offer service
    2014-12-22 12:08:33,001 INFO org.apache.hadoop.ipc.Server: IPC Server Responder: starting
    2014-12-22 12:08:33,003 INFO org.apache.hadoop.ipc.Server: IPC Server listener on 50020: starting
    2014-12-22 12:08:33,536 INFO org.apache.hadoop.hdfs.server.common.Storage: DataNode version: -56 and NameNode layout version: -60
    
    2014-12-22 12:08:33,699 INFO org.apache.hadoop.hdfs.server.common.Storage: Lock on /home/hadoop/tmp/dfs/data/in_use.lock acquired by nodename 17247@henry-ThinkPad-T400
    2014-12-22 12:08:33,706 FATAL org.apache.hadoop.hdfs.server.datanode.DataNode: Initialization failed for Block pool <registering> (Datanode Uuid unassigned) service to localhost/127.0.0.1:8020. Exiting.
    java.io.IOException: Incompatible clusterIDs in /home/hadoop/tmp/dfs/data: namenode clusterID = CID-19f887ba-2e8d-4c7e-ae01-e38a30581693; datanode clusterID = CID-14aac0b3-3c32-45db-adb8-b5fc494eaa3d
            at org.apache.hadoop.hdfs.server.datanode.DataStorage.doTransition(DataStorage.java:646)
            at org.apache.hadoop.hdfs.server.datanode.DataStorage.addStorageLocations(DataStorage.java:320)
            at org.apache.hadoop.hdfs.server.datanode.DataStorage.recoverTransitionRead(DataStorage.java:403)
            at org.apache.hadoop.hdfs.server.datanode.DataStorage.recoverTransitionRead(DataStorage.java:422)
            at org.apache.hadoop.hdfs.server.datanode.DataNode.initStorage(DataNode.java:1311)
            at org.apache.hadoop.hdfs.server.datanode.DataNode.initBlockPool(DataNode.java:1276)
            at org.apache.hadoop.hdfs.server.datanode.BPOfferService.verifyAndSetNamespaceInfo(BPOfferService.java:314)
            at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.connectToNNAndHandshake(BPServiceActor.java:220)
            at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:828)
            at java.lang.Thread.run(Thread.java:662)
    2014-12-22 12:08:33,716 WARN org.apache.hadoop.hdfs.server.datanode.DataNode: Ending block pool service for: Block pool <registering> (Datanode Uuid unassigned) service to localhost/127.0.0.1:8020
    2014-12-22 12:08:33,718 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Removed Block pool <registering> (Datanode Uuid unassigned)
    2014-12-22 12:08:35,718 WARN org.apache.hadoop.hdfs.server.datanode.DataNode: Exiting Datanode
    2014-12-22 12:08:35,720 INFO org.apache.hadoop.util.ExitUtil: Exiting with status 0
    2014-12-22 12:08:35,722 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: SHUTDOWN_MSG:
    /************************************************************
    SHUTDOWN_MSG: Shutting down DataNode at henry-ThinkPad-T400/127.0.0.1
    ************************************************************/

    问题分析:在第一次格式化dfs后,启动并使用了hadoop,后来又重新执行了格式化命令(hdfs namenode -format),

         这时namenode的clusterID会重新生成,而datanode的clusterID 保持不变。

    问题总结:datanode的clusterID 和 namenode的clusterID 不匹配。

    解决办法:根据日志中的路径,cd /home/hadoop/tmp/dfs 能看到 data和name两个文件夹,

         将name/current下的VERSION中的clusterID复制到data/current下的VERSION中,覆盖掉原来的clusterID,让两个保持一致

         然后重启,启动后执行jps,查看进程

  • 相关阅读:
    揺する、揺らす、揺さぶる区別
    test
    W5N1のW2D2
    微软企业库6的使用方法
    Delphi XE6 原生解析json
    delphi URL 编码的转换
    Go语言配置与开发环境配置
    Win7 IIS下启用ASP.NET
    Windows2003远程桌面单会话登录
    IIS7应用程序池集成和经典的区别
  • 原文地址:https://www.cnblogs.com/haimishasha/p/7629943.html
Copyright © 2011-2022 走看看