zoukankan      html  css  js  c++  java
  • How to solve java.net.SocketTimeoutException:60000millis problem in HDFS

    Many HDFS users encounter the following error when DFSClient ready file from a certain Data Node.

            "Java.net.SocketTimeoutException: 60000millis timeout while waiting for channel to be ready for read. ch"

    The reason about this error is that the Data Node fails to communicate with DFSClient when DFSClient requests some block from that Data Node, which often occurs when the small cluster conntains too many files and the Data Node with such heavy burden cannot schedules a certain block within the limited time (for instance 60 sec). Then DFSClient waits untill this interval fires, than turn to request the other Data Node containing the same block.

    If HDFS is used for in-time service, people wil find the 60 sec timeout is too long and they hope to shorten the interval making the DFSClient switch to other Data Node quickly. When I encounter is problem, I seach the Internet for solution. Depressed, No answeris correct.

    Then I debug the source code, finding the timeout value is controlled by the config item "ipc.ping.interval". This item has the default value 60000 (millisec). You can add this config item into you"core-site.xml" file and it will take effert when DFSClient starts.

  • 相关阅读:
    什么是内卷?
    iphone与PC端如何传BUG截图
    java应用服务占用cpu过高,如何优化
    性能测试常见问题FAQ
    性能测试工程师能力进阶三部曲
    jmeter分布式压测试部署
    了解token及分类
    常见端口号及其服务
    2714
    python
  • 原文地址:https://www.cnblogs.com/peizhe123/p/5540825.html
Copyright © 2011-2022 走看看