zoukankan      html  css  js  c++  java
  • hbase-indexer官网wiki

    Home

     
    Gabriel Reid edited this page on 7 Aug 2013 · 8 revisions

    Introduction

    The HBase Indexer project provides indexing (via Solr) for content stored in HBase. It provides a flexible and extensible way of defining indexing rules, and is designed to scale.

    Indexing is performed asynchronously, so it does not impact write throughput on HBase. SolrCloud is used for storing the actual index in order to ensure scalability of the indexing.

    Getting started with the HBase Indexer

    1. Make sure you've got the required software installed, as detailed on the Requirements page.
    2. Follow the Tutorial to get a feel for how to use the HBase Indexer.
    3. Customize your indexing setup as needed using the other reference documentation provided here.

    How it works

    The HBase Indexer works by acting as an HBase replication sink. As updates are written to HBase region servers, they are "replicated" asynchronously to the HBase Indexer processes.

    The indexer analyzes incoming HBase mutation events, and where applicable it creates Solr documents and pushes them to SolrCloud servers.

    The indexed documents in Solr contain enough information to uniquely identify the HBase row that they are based on, allowing you to use Solr to search for content that is stored in HBase.

    HBase replication is based on reading the HBase log files, which are the precise source of truth of the what is stored in HBase: there are no missing or no extra events. In various cases, the log also contains all the information needed to index, so that no expensive random-read on HBase is necessary (see the read-row attribute in the Indexer Configuration).

    HBase replication delivers (small) batches of events. HBase-indexer exploits this by avoiding double-indexing of the same row if it would have been updated twice in a short time frame, and as well will batch/buffer the updates towards Solr, which gives important performance gains. The updates are applied to Solr before confirming the processing of the events to HBase, so that no event loss is possible.

    Horizontal scalability

    All information about indexers is stored in ZooKeeper. New indexer hosts can always be added to a cluster, in the same way that HBase regionservers can be added to to an HBase cluster.

    All indexing work for a single configured indexer is shared over all machines in the cluster. In this way, adding additional indexer nodes allows horizontal scaling.

    Automatic failure handling

    The HBase replication system upon which the HBase Indexer is based is designed to handle hardware failures. Because the HBase Indexer is based on this system, it also benefits from the same ability to handle failures.

    In general, indexing nodes going down or Solr nodes going down will not result in any lost data in the HBase Indexer.

  • 相关阅读:
    python 3.x 不再提供raw_print()
    php中fileatim,filectime和filemtime函数的区别
    如何将文本以BLOB类型存入数据库并取出
    tomcat启动不了——Error initializing endpoint——java.net.BindException: Address already in use: JVM_Bind
    hdu 2188(巴什博弈入门 )
    hdu 2187(贪心)
    Sentinel数据处理工具包SNAP Python开发环境搭建
    conda 安装包命令
    使用SSH连接WSL
    win10 安装WSL 出现 WslRegisterDistribution failed with error: 0x8000000d
  • 原文地址:https://www.cnblogs.com/felixzh/p/11389465.html
Copyright © 2011-2022 走看看