zoukankan      html  css  js  c++  java
  • hadoop报错:hdfs.DFSClient: Exception in createBlockOutputStream

    hadoop跑任务搞的好好的,后来把hadoop-dir移了一个位置,结果报错:

    java.io.EOFException: Premature EOF: no length prefix available
            at org.apache.hadoop.hdfs.protocolPB.PBHelper.vintPrefixed(PBHelper.java:1492)
            at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.createBlockOutputStream(DFSOutputStream.java:1155)
            at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.nextBlockOutputStream(DFSOutputStream.java:1088)
            at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:514)
    15/03/24 18:26:40 INFO hdfs.DFSClient: Abandoning BP-1909118226-192.168.19.234-1427110524238:blk_1073762363_21550
    15/03/24 18:26:40 INFO hdfs.DFSClient: Excluding datanode 192.168.21.24:50010
    copy from: /root/zenggq/jn2/data2w/t0.head_2000 to /recom1000/t0.head_2000
    15/03/24 18:26:41 INFO hdfs.DFSClient: Exception in createBlockOutputStream
    java.io.EOFException: Premature EOF: no length prefix available
            at org.apache.hadoop.hdfs.protocolPB.PBHelper.vintPrefixed(PBHelper.java:1492)
            at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.createBlockOutputStream(DFSOutputStream.java:1155)
            at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.nextBlockOutputStream(DFSOutputStream.java:1088)
            at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:514)
    15/03/24 18:26:41 INFO hdfs.DFSClient: Abandoning BP-1909118226-192.168.19.234-1427110524238:blk_1073762365_21552
    15/03/24 18:26:41 INFO hdfs.DFSClient: Excluding datanode 192.168.21.23:50010
    15/03/24 18:26:41 INFO hdfs.DFSClient: Exception in createBlockOutputStream
    java.io.IOException: Bad connect ack with firstBadLink as 192.168.21.24:50010
            at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.createBlockOutputStream(DFSOutputStream.java:1166)
            at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.nextBlockOutputStream(DFSOutputStream.java:1088)
            at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:514)
    15/03/24 18:26:41 INFO hdfs.DFSClient: Abandoning BP-1909118226-192.168.19.234-1427110524238:blk_1073762366_21553
    15/03/24 18:26:41 INFO hdfs.DFSClient: Excluding datanode 192.168.21.24:50010
    15/03/24 18:26:41 INFO hdfs.DFSClient: Exception in createBlockOutputStream
    java.io.EOFException: Premature EOF: no length prefix available
            at org.apache.hadoop.hdfs.protocolPB.PBHelper.vintPrefixed(PBHelper.java:1492)
            at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.createBlockOutputStream(DFSOutputStream.java:1155)
            at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.nextBlockOutputStream(DFSOutputStream.java:1088)
            at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:514)
    15/03/24 18:26:41 INFO hdfs.DFSClient: Abandoning BP-1909118226-192.168.19.234-1427110524238:blk_1073762367_21554
    15/03/24 18:26:41 INFO hdfs.DFSClient: Excluding datanode 192.168.19.236:50010
    15/03/24 18:26:41 INFO hdfs.DFSClient: Exception in createBlockOutputStream
    java.io.EOFException: Premature EOF: no length prefix available
            at org.apache.hadoop.hdfs.protocolPB.PBHelper.vintPrefixed(PBHelper.java:1492)
            at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.createBlockOutputStream(DFSOutputStream.java:1155)
            at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.nextBlockOutputStream(DFSOutputStream.java:1088)
            at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:514)
    15/03/24 18:26:41 INFO hdfs.DFSClient: Abandoning BP-1909118226-192.168.19.234-1427110524238:blk_1073762368_21555
    15/03/24 18:26:41 INFO hdfs.DFSClient: Excluding datanode 192.168.21.30:50010
    15/03/24 18:26:41 WARN hdfs.DFSClient: DataStreamer Exception
    java.io.IOException: Unable to create new block.
            at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.nextBlockOutputStream(DFSOutputStream.java:1100)
            at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:514)
    15/03/24 18:26:41 WARN hdfs.DFSClient: Could not get block locations. Source file "/recom1000/t1.head_2000" - Aborting...
    Exception in thread "main" java.io.EOFException: Premature EOF: no length prefix available
            at org.apache.hadoop.hdfs.protocolPB.PBHelper.vintPrefixed(PBHelper.java:1492)
            at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.createBlockOutputStream(DFSOutputStream.java:1155)
            at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.nextBlockOutputStream(DFSOutputStream.java:1088)
            at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:514)
    15/03/24 18:26:41 ERROR hdfs.DFSClient: Failed to close file /recom1000/t1.head_2000
    java.io.EOFException: Premature EOF: no length prefix available
            at org.apache.hadoop.hdfs.protocolPB.PBHelper.vintPrefixed(PBHelper.java:1492)
            at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.createBlockOutputStream(DFSOutputStream.java:1155)
            at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.nextBlockOutputStream(DFSOutputStream.java:1088)
            at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:514)
    [root@master jn2]#

    找了一圈答案没有,一说是datanode进程没有,而说是防火墙没关。结果我这两个都没问题。

    后来我把hadoop-dir下的data目录删了。然后重新格式化namenode

    hadoop namenode -format

    然后的然后就是好了

    博上文章,均属原创,如需转载,请注明出处
  • 相关阅读:
    数据结构(复习)链表完结篇
    第三部分_JSP详解续
    第二部分_搭建Java Web开发环境与配置Tomcat服务器&JSP详解
    第一部分_HTTP协议详解&HTML常用控件
    集合框架中的接口及其实现类
    封底估算
    从起泡排序探究算法正确性证明的一般规律
    各种曲线运动、弹球、笔记
    Android 之px于dp在Java代码中的转换
    sqlite之聚合函数的使用
  • 原文地址:https://www.cnblogs.com/whu-zeng/p/4363706.html
Copyright © 2011-2022 走看看