zoukankan      html  css  js  c++  java
  • How to solve java.net.SocketTimeoutException:60000millis problem in HDFS

    Many HDFS users encounter the following error when DFSClient ready file from a certain Data Node.

            "Java.net.SocketTimeoutException: 60000millis timeout while waiting for channel to be ready for read. ch"

    The reason about this error is that the Data Node fails to communicate with DFSClient when DFSClient requests some block from that Data Node, which often occurs when the small cluster conntains too many files and the Data Node with such heavy burden cannot schedules a certain block within the limited time (for instance 60 sec). Then DFSClient waits untill this interval fires, than turn to request the other Data Node containing the same block.

    If HDFS is used for in-time service, people wil find the 60 sec timeout is too long and they hope to shorten the interval making the DFSClient switch to other Data Node quickly. When I encounter is problem, I seach the Internet for solution. Depressed, No answeris correct.

    Then I debug the source code, finding the timeout value is controlled by the config item "ipc.ping.interval". This item has the default value 60000 (millisec). You can add this config item into you"core-site.xml" file and it will take effert when DFSClient starts.

  • 相关阅读:
    2333
    STL string
    后缀自动机的应用
    省选一轮
    等价类计数问题(Polya定理和burnside引理)
    Prufer序列与树的计数(坑)
    分治算法
    生成函数
    莫队算法
    Xamarin 技术解析
  • 原文地址:https://www.cnblogs.com/peizhe123/p/5540825.html
Copyright © 2011-2022 走看看