zoukankan      html  css  js  c++  java
  • hadoop多次格式化后,导致datanode启动不了

    hadoop namenode -format
    多次格式化后,datanode启动不了

    org.apache.hadoop.ipc.RemoteException(java.io.IOException): File /user/beifeng/core-site.xml._COPYING_ could only be replicated to 0 nodes instead of minReplication (=1).  There are 0 datanode(s) running and no node(s) are excluded in this operation.
            at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.chooseTarget(BlockManager.java:1471)
            at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:2791)
            at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.addBlock(NameNodeRpcServer.java:606)
            at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.addBlock(ClientNamenodeProtocolServerSideTranslatorPB.java:455)
            at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
            at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:585)
            at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:928)
            at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2013)
            at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2009)
            at java.security.AccessController.doPrivileged(Native Method)
            at javax.security.auth.Subject.doAs(Subject.java:415)
            at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
            at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2007)
    
            at org.apache.hadoop.ipc.Client.call(Client.java:1411)
            at org.apache.hadoop.ipc.Client.call(Client.java:1364)
            at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:206)
            at com.sun.proxy.$Proxy14.addBlock(Unknown Source)
            at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
            at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
            at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
            at java.lang.reflect.Method.invoke(Method.java:606)
            at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:187)
            at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102)
            at com.sun.proxy.$Proxy14.addBlock(Unknown Source)
            at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.addBlock(ClientNamenodeProtocolTranslatorPB.java:368)
            at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.locateFollowingBlock(DFSOutputStream.java:1449)
            at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.nextBlockOutputStream(DFSOutputStream.java:1270)
            at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:526)
    put: File /user/beifeng/core-site.xml._COPYING_ could only be replicated to 0 nodes instead of minReplication (=1).  There are 0 datanode(s) running and no node(s) are excluded in this operation.

    0.20.2版本解决方式:
    1、查看名称节点上(即在配置文件中配置的hadoop.tmp.dir参数路径)/usr/hadoop0.20.2/hadoop-huanghz/dfs/name/current/文件夹中VERSION文件中的namespaceid;
    2、在两个数据节点修改上dfs.data.dir配置的路径下current文件夹中的VERSION文件namespaceid与名称节点保持一致

    2.20版本解决
    /data/hadoop/dfs/name/current/VERSION
    用name下面的clusterID,修改datanode的/data/hadoop/dfs/data/current/VERSION 里面的clusterID
    每次格式化,name下面的VERSION的clusterID会产生一个新的ID,要去修改各个节点的VERSION的clusterID 

    ……/dfs/data/current


    storageID=DS-1959445666-10.161.138.100-50010-1386493413986
    clusterID=CID-64a3a726-29e4-4d80-86a6-035ef33a225b
    cTime=0
    storageType=DATA_NODE
    layoutVersion=-47

    ……/dfs/name/current/VERSION
    #Fri Jan 03 10:37:48 CST 2014
    namespaceID=1667984727
    clusterID=CID-42c6d540-c3ca-44df-95e8-01a6d87effb5
    cTime=0
    storageType=NAME_NODE
    blockpoolID=BP-220196921-10.161.138.100-1388716668863
    layoutVersion=-47

  • 相关阅读:
    Vue 第六章 组件切换component、transition-group渲染
    Vue 第五章 钩子函数实现半场动画
    Vue 第四章 动画效果、animate第三方插件动画效果
    Vue 第三章 使用vue-resource列表案例、全局根域名配置
    hive介绍
    Hive性能优化--map数和reduce数
    Storm实现实时大数据分析
    hive架构原理简析-mapreduce部分
    hive入门学习线路指导
    HBase基本概念
  • 原文地址:https://www.cnblogs.com/yeahwell/p/5642798.html
Copyright © 2011-2022 走看看