zoukankan      html  css  js  c++  java
  • DHT(Distributed Hash Table) Translator

    DHT(Distributed Hash Table) Translator

    What is DHT?

    DHT is the real core of how GlusterFS aggregates capacity and performance across multiple servers. Its responsibility is to place each file on exactly one of its subvolumes – unlike either replication (which places copies on all of its subvolumes) or striping (which places pieces onto all of its subvolumes). It’s a routing function, not splitting or copying.

    How DHT works?

    The basic method used in DHT is consistent hashing. Each subvolume (brick) is assigned a range within a 32-bit hash space, covering the entire range with no holes or overlaps. Then each file is also assigned a value in that same space, by hashing its name. Exactly one brick will have an assigned range including the file’s hash value, and so the file “should” be on that brick. However, there are many cases where that won’t be the case, such as when the set of bricks (and therefore the range assignment of ranges) has changed since the file was created, or when a brick is nearly full. Much of the complexity in DHT involves these special cases, which we’ll discuss in a moment.

    When you open() a file, the distribute translator is giving one piece of information to find your file, the file-name. To determine where that file is, the translator runs the file-name through a hashing algorithm in order to turn that file-name into a number.

    Few Observations of DHT hash-values assignment:

    • The assignment of hash ranges to bricks is determined by extended attributes stored on directories, hence distribution is directory-specific.

    • Consistent hashing is usually thought of as hashing around a circle, but in GlusterFS it’s more linear. There’s no need to “wrap around” at zero, because there’s always a break (between one brick’s range and another’s) at zero.

    • If a brick is missing, there will be a hole in the hash space. Even worse, if hash ranges are reassigned while a brick is offline, some of the new ranges might overlap with the (now out of date) range stored on that brick, creating a bit of confusion about where files should be.

  • 相关阅读:
    资源管理与调度系统-资源管理系统Mesos
    hive介绍及架构设计
    企业微信号注册步骤详解及zabbix报警媒介配置
    zabbix Server 4.0 监控Nginx
    zabbix Server 4.0 监控TCP的12种状态
    资源管理与调度系统-YARN资源隔离及以YARN为核心的生态系统
    资源管理与调度系统-YARN的资源调度器
    资源管理与调度系统-YARN的基本架构与原理
    批处理引擎MapReduce应用案例
    批处理引擎MapReduce内部原理
  • 原文地址:https://www.cnblogs.com/vman/p/5081046.html
Copyright © 2011-2022 走看看