zoukankan      html  css  js  c++  java
  • hadoop balancer 二

    1.每次迭代一个datanode会移动不超过如下两者中较小的一个的数据量

       1)10G

       2)能力阈值 dfs.datanode.balance.bandwidthPerSec

      每次迭代时间不会超过20分钟

    <p>The tool moves blocks from highly utilized datanodes to poorly
     * utilized datanodes iteratively. In each iteration a datanode moves or
     * receives no more than the lesser of 10G bytes or the threshold fraction
     * of its capacity. Each iteration runs no more than 20 minutes.
     * At the end of each iteration, the balancer obtains updated datanodes
     * information from the namenode.

    2.balancer自动停止的情景

     * <p>The balancer automatically exits when any of the following five
     * conditions is satisfied:
     * <ol>
     * <li>The cluster is balanced;
     * <li>No block can be moved;
     * <li>No block has been moved for five consecutive iterations;
     * <li>An IOException occurs while communicating with the namenode;
     * <li>Another balancer is running.
     * </ol>

     <p>Upon exit, a balancer returns an exit code and prints one of the
     * following messages to the output file in corresponding to the above exit
     * reasons:
     * <ol>
     * <li>The cluster is balanced. Exiting
     * <li>No block can be moved. Exiting...
     * <li>No block has been moved for 5 iterations. Exiting...
     * <li>Received an IO exception: failure reason. Exiting...
     * <li>Another balancer is running. Exiting...
     * </ol>

    3.一个datanode同时可用于balance的移动最大数

     /** The maximum number of concurrent blocks moves for
       * balancing purpose at a datanode
       */
      public static final int MAX_NUM_CONCURRENT_MOVES = 5;
      private static final int MAX_NO_PENDING_BLOCK_ITERATIONS = 5;
     

    4.

  • 相关阅读:
    sersync+rsync原理及部署
    rsync同步
    zabbix 3.2.2 agent端(源码包)安装部署
    zabbix配置
    Netbackup media server部署报错
    Oracle_rac命令
    Linux系统克隆为iso镜像盘(类似win gost)
    Nebackup清除磁带数据重新使用
    V7000初始化
    【数据案例】服务器崩溃后的数据恢复方法
  • 原文地址:https://www.cnblogs.com/roger888/p/6703742.html
Copyright © 2011-2022 走看看