zoukankan      html  css  js  c++  java
  • hadoop balancer 二

    1.每次迭代一个datanode会移动不超过如下两者中较小的一个的数据量

       1)10G

       2)能力阈值 dfs.datanode.balance.bandwidthPerSec

      每次迭代时间不会超过20分钟

    <p>The tool moves blocks from highly utilized datanodes to poorly
     * utilized datanodes iteratively. In each iteration a datanode moves or
     * receives no more than the lesser of 10G bytes or the threshold fraction
     * of its capacity. Each iteration runs no more than 20 minutes.
     * At the end of each iteration, the balancer obtains updated datanodes
     * information from the namenode.

    2.balancer自动停止的情景

     * <p>The balancer automatically exits when any of the following five
     * conditions is satisfied:
     * <ol>
     * <li>The cluster is balanced;
     * <li>No block can be moved;
     * <li>No block has been moved for five consecutive iterations;
     * <li>An IOException occurs while communicating with the namenode;
     * <li>Another balancer is running.
     * </ol>

     <p>Upon exit, a balancer returns an exit code and prints one of the
     * following messages to the output file in corresponding to the above exit
     * reasons:
     * <ol>
     * <li>The cluster is balanced. Exiting
     * <li>No block can be moved. Exiting...
     * <li>No block has been moved for 5 iterations. Exiting...
     * <li>Received an IO exception: failure reason. Exiting...
     * <li>Another balancer is running. Exiting...
     * </ol>

    3.一个datanode同时可用于balance的移动最大数

     /** The maximum number of concurrent blocks moves for
       * balancing purpose at a datanode
       */
      public static final int MAX_NUM_CONCURRENT_MOVES = 5;
      private static final int MAX_NO_PENDING_BLOCK_ITERATIONS = 5;
     

    4.

  • 相关阅读:
    分布式任务调度 xxl-job
    【线上】 select * from xxoo where 1=1应用挂掉
    【死磕ES】七、基础检索
    【死磕ES】四、基本操作
    【死磕ES】三、基本概念
    【死磕ES】二、搭建环境
    Mac共享文件夹
    微信小程序下拉刷新,上拉加载
    微信小程序textarea输入框出现[object Object]
    微信小程序official-account的使用
  • 原文地址:https://www.cnblogs.com/roger888/p/6703742.html
Copyright © 2011-2022 走看看