zoukankan      html  css  js  c++  java
  • namenode和datanode启动失败

    1.namenode启动失败,查看错误原因,是无法格式化,再看日志,根据日志提示,清空对应的目录,即可解决这个问题。

    2.datanode启动失败:

    Can't open /var/run/cloudera-scm-agent/process/261-hdfs-DATANODE/supervisor.conf: Permission denied.
    

    cd到/var/run/cloudera-scm-agent/process/261-hdfs-DATANODE目录下,看到:

    -rw------- 1 root root 3146  2月  5 18:03 2018 supervisor.conf

    怀疑是权限问题,上网查询了之后,才知道看起来是权限问题,其实不是,需要看看具体的datanode启动日志,于是到/var/log/hadoop-hdfs/目录下,清空目录,重新启动一下datanode,查看新生成的log日志,如下:

    2018-02-05 18:29:10,370 WARN org.apache.hadoop.hdfs.server.common.Storage: java.io.IOException: Incompatible clusterIDs in /dfs/dn: namenode clusterID = cluster10; datanode
     clusterID = cluster20
    2018-02-05 18:29:10,371 FATAL org.apache.hadoop.hdfs.server.datanode.DataNode: Initialization failed for Block pool <registering> (Datanode Uuid 5c80b393-7d62-4016-bafa-c44
    63af7a45f) service to iot1/172.30.1.245:8022. Exiting. 
    java.io.IOException: All specified directories are failed to load.
            at org.apache.hadoop.hdfs.server.datanode.DataStorage.recoverTransitionRead(DataStorage.java:478)
            at org.apache.hadoop.hdfs.server.datanode.DataNode.initStorage(DataNode.java:1394)
            at org.apache.hadoop.hdfs.server.datanode.DataNode.initBlockPool(DataNode.java:1355)
            at org.apache.hadoop.hdfs.server.datanode.BPOfferService.verifyAndSetNamespaceInfo(BPOfferService.java:317)
            at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.connectToNNAndHandshake(BPServiceActor.java:228)
            at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:829)
            at java.lang.Thread.run(Thread.java:745)
    2018-02-05 18:29:10,378 WARN org.apache.hadoop.hdfs.server.datanode.DataNode: Ending block pool service for: Block pool <registering> (Datanode Uuid 5c80b393-7d62-4016-bafa
    -c4463af7a45f) service to iot1/172.30.1.245:8022
    

      

    日志里有这样一行:

    java.io.IOException: Incompatible clusterIDs in /dfs/dn: namenode clusterID = cluster10; datanode clusterID = cluster20

    这就是namenode的clientID与datanode的clientID不一致的错误。

    解决办法就是把datanode的clientID改的和namenode一致

    datanode的clientID记录在/dfs/dn/current/VERSION文件中(即配置文件中dfs.data.dir在本地系统的路径的current/VERSION文件,dfs.data.dir这个目录可以在ClouderaManager的管理页面查找到)

    最后修改的一致就解决问题了

    # 打可执行jar包mvn package assembly:single
    Can't open /var/run/cloudera-scm-agent/process/261-hdfs-DATANODE/supervisor.conf: Permission denied.
    cd到/var/run/cloudera-scm-agent/process/261-hdfs-DATANODE目录下,看到:

  • 相关阅读:
    yii2 模型查询使用计算值
    git回滚操作
    yii2 模型搜索时 or 条件查询
    从一台服务器发送文件到另一台服务器
    学习swoft的第三天_AOP切面
    C面试题汇总(转)
    秒杀多线程:多线程笔试面试题汇总(转)
    链表的常见操作(转)
    YAFFS跟踪
    libusb开发指南
  • 原文地址:https://www.cnblogs.com/gaojiang/p/8418780.html
Copyright © 2011-2022 走看看