zoukankan      html  css  js  c++  java
  • hbase region与内存的关系

    http://hbase.apache.org/book.html#ops.capacity.regions.count

    In production scenarios, where you have a lot of data, you are normally concerned with the maximum number of regions you can have per server. too many regions has technical discussion on the subject. Basically, the maximum number of regions is mostly determined by memstore memory usage. Each region has its own memstores; these grow up to a configurable size; usually in 128-256 MB range, see hbase.hregion.memstore.flush.size. One memstore exists per column family (so there’s only one per region if there’s one CF in the table). The RS dedicates some fraction of total memory to its memstores (see hbase.regionserver.global.memstore.size). If this memory is exceeded (too much memstore usage), it can cause undesirable consequences such as unresponsive server or compaction storms. A good starting point for the number of regions per RS (assuming one table) is:

    ((RS memory) * (total memstore fraction)) / ((memstore size)*(# column families))

    This formula is pseudo-code. Here are two formulas using the actual tunable parameters, first for HBase 0.98+ and second for HBase 0.94.x.

    HBase 0.98.x
    ((RS Xmx) * hbase.regionserver.global.memstore.size) / (hbase.hregion.memstore.flush.size * (# column families))
    HBase 0.94.x
    ((RS Xmx) * hbase.regionserver.global.memstore.upperLimit) / (hbase.hregion.memstore.flush.size * (# column families))+

    If a given RegionServer has 16 GB of RAM, with default settings, the formula works out to 16384*0.4/128 ~ 51 regions per RS is a starting point. The formula can be extended to multiple tables; if they all have the same configuration, just use the total number of families.

  • 相关阅读:
    信息安全系统设计基础学习总结第二周
    一维数组求和2
    求一维数组的最大子数组的和
    第三周学习进度
    构建之法阅读笔记二
    四则运算三
    学习进度第二周
    四则运算二
    测试第二次程序
    阅读计划
  • 原文地址:https://www.cnblogs.com/jixiangxiang/p/5520871.html
Copyright © 2011-2022 走看看