zoukankan      html  css  js  c++  java
  • hadoop报错:hdfs.DFSClient: Exception in createBlockOutputStream

    hadoop跑任务搞的好好的,后来把hadoop-dir移了一个位置,结果报错:

    java.io.EOFException: Premature EOF: no length prefix available
            at org.apache.hadoop.hdfs.protocolPB.PBHelper.vintPrefixed(PBHelper.java:1492)
            at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.createBlockOutputStream(DFSOutputStream.java:1155)
            at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.nextBlockOutputStream(DFSOutputStream.java:1088)
            at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:514)
    15/03/24 18:26:40 INFO hdfs.DFSClient: Abandoning BP-1909118226-192.168.19.234-1427110524238:blk_1073762363_21550
    15/03/24 18:26:40 INFO hdfs.DFSClient: Excluding datanode 192.168.21.24:50010
    copy from: /root/zenggq/jn2/data2w/t0.head_2000 to /recom1000/t0.head_2000
    15/03/24 18:26:41 INFO hdfs.DFSClient: Exception in createBlockOutputStream
    java.io.EOFException: Premature EOF: no length prefix available
            at org.apache.hadoop.hdfs.protocolPB.PBHelper.vintPrefixed(PBHelper.java:1492)
            at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.createBlockOutputStream(DFSOutputStream.java:1155)
            at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.nextBlockOutputStream(DFSOutputStream.java:1088)
            at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:514)
    15/03/24 18:26:41 INFO hdfs.DFSClient: Abandoning BP-1909118226-192.168.19.234-1427110524238:blk_1073762365_21552
    15/03/24 18:26:41 INFO hdfs.DFSClient: Excluding datanode 192.168.21.23:50010
    15/03/24 18:26:41 INFO hdfs.DFSClient: Exception in createBlockOutputStream
    java.io.IOException: Bad connect ack with firstBadLink as 192.168.21.24:50010
            at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.createBlockOutputStream(DFSOutputStream.java:1166)
            at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.nextBlockOutputStream(DFSOutputStream.java:1088)
            at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:514)
    15/03/24 18:26:41 INFO hdfs.DFSClient: Abandoning BP-1909118226-192.168.19.234-1427110524238:blk_1073762366_21553
    15/03/24 18:26:41 INFO hdfs.DFSClient: Excluding datanode 192.168.21.24:50010
    15/03/24 18:26:41 INFO hdfs.DFSClient: Exception in createBlockOutputStream
    java.io.EOFException: Premature EOF: no length prefix available
            at org.apache.hadoop.hdfs.protocolPB.PBHelper.vintPrefixed(PBHelper.java:1492)
            at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.createBlockOutputStream(DFSOutputStream.java:1155)
            at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.nextBlockOutputStream(DFSOutputStream.java:1088)
            at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:514)
    15/03/24 18:26:41 INFO hdfs.DFSClient: Abandoning BP-1909118226-192.168.19.234-1427110524238:blk_1073762367_21554
    15/03/24 18:26:41 INFO hdfs.DFSClient: Excluding datanode 192.168.19.236:50010
    15/03/24 18:26:41 INFO hdfs.DFSClient: Exception in createBlockOutputStream
    java.io.EOFException: Premature EOF: no length prefix available
            at org.apache.hadoop.hdfs.protocolPB.PBHelper.vintPrefixed(PBHelper.java:1492)
            at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.createBlockOutputStream(DFSOutputStream.java:1155)
            at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.nextBlockOutputStream(DFSOutputStream.java:1088)
            at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:514)
    15/03/24 18:26:41 INFO hdfs.DFSClient: Abandoning BP-1909118226-192.168.19.234-1427110524238:blk_1073762368_21555
    15/03/24 18:26:41 INFO hdfs.DFSClient: Excluding datanode 192.168.21.30:50010
    15/03/24 18:26:41 WARN hdfs.DFSClient: DataStreamer Exception
    java.io.IOException: Unable to create new block.
            at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.nextBlockOutputStream(DFSOutputStream.java:1100)
            at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:514)
    15/03/24 18:26:41 WARN hdfs.DFSClient: Could not get block locations. Source file "/recom1000/t1.head_2000" - Aborting...
    Exception in thread "main" java.io.EOFException: Premature EOF: no length prefix available
            at org.apache.hadoop.hdfs.protocolPB.PBHelper.vintPrefixed(PBHelper.java:1492)
            at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.createBlockOutputStream(DFSOutputStream.java:1155)
            at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.nextBlockOutputStream(DFSOutputStream.java:1088)
            at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:514)
    15/03/24 18:26:41 ERROR hdfs.DFSClient: Failed to close file /recom1000/t1.head_2000
    java.io.EOFException: Premature EOF: no length prefix available
            at org.apache.hadoop.hdfs.protocolPB.PBHelper.vintPrefixed(PBHelper.java:1492)
            at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.createBlockOutputStream(DFSOutputStream.java:1155)
            at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.nextBlockOutputStream(DFSOutputStream.java:1088)
            at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:514)
    [root@master jn2]#

    找了一圈答案没有,一说是datanode进程没有,而说是防火墙没关。结果我这两个都没问题。

    后来我把hadoop-dir下的data目录删了。然后重新格式化namenode

    hadoop namenode -format

    然后的然后就是好了

    博上文章,均属原创,如需转载,请注明出处
  • 相关阅读:
    第八章 Libgdx输入处理(7)罗盘
    第六日
    第十日
    第五日
    第七日
    第九日
    第四日
    第三日
    使用EVM进行项目管理时的注意事项
    第八日
  • 原文地址:https://www.cnblogs.com/whu-zeng/p/4363706.html
Copyright © 2011-2022 走看看