zoukankan      html  css  js  c++  java
  • hbase与hive集成:hive读取hbase中数据

    1、创建hbase jar包到hive lib目录软连接

    hive需要jar包:
    hive-hbase-handler-0.13.1-cdh5.3.6.jar
    zookeeper-3.4.5-cdh5.3.6.jar
    guava-12.0.1.jar                  --删除hive下低版本jar包,cp hbase中的到hive/lib/下(即需要注意jar包版本)

    ln -s /opt/cdh-5.3.6/hbase-0.98.6-cdh5.3.6/lib/hbase-server-0.98.6-cdh5.3.6.jar /opt/cdh-5.3.6/hive-0.13.1-cdh5.3.6/lib/hbase-server-0.98.6-cdh5.3.6.jar
    ln -s /opt/cdh-5.3.6/hbase-0.98.6-cdh5.3.6/lib/hbase-client-0.98.6-cdh5.3.6.jar /opt/cdh-5.3.6/hive-0.13.1-cdh5.3.6/lib/hbase-client-0.98.6-cdh5.3.6.jar
    ln -s /opt/cdh-5.3.6/hbase-0.98.6-cdh5.3.6/lib/hbase-protocol-0.98.6-cdh5.3.6.jar /opt/cdh-5.3.6/hive-0.13.1-cdh5.3.6/lib/hbase-protocol-0.98.6-cdh5.3.6.jar
    ln -s /opt/cdh-5.3.6/hbase-0.98.6-cdh5.3.6/lib/hbase-it-0.98.6-cdh5.3.6.jar /opt/cdh-5.3.6/hive-0.13.1-cdh5.3.6/lib/hbase-it-0.98.6-cdh5.3.6.jar
    ln -s /opt/cdh-5.3.6/hbase-0.98.6-cdh5.3.6/lib/htrace-core-2.04.jar /opt/cdh-5.3.6/hive-0.13.1-cdh5.3.6/lib/htrace-core-2.04.jar
    ln -s /opt/cdh-5.3.6/hbase-0.98.6-cdh5.3.6/lib/hbase-hadoop2-compat-0.98.6-cdh5.3.6.jar /opt/cdh-5.3.6/hive-0.13.1-cdh5.3.6/lib/hbase-hadoop2-compat-0.98.6-cdh5.3.6.jar
    ln -s /opt/cdh-5.3.6/hbase-0.98.6-cdh5.3.6/lib/hbase-hadoop-compat-0.98.6-cdh5.3.6.jar /opt/cdh-5.3.6/hive-0.13.1-cdh5.3.6/lib/hbase-hadoop-compat-0.98.6-cdh5.3.6.jar
    ln -s /opt/cdh-5.3.6/hbase-0.98.6-cdh5.3.6/lib/high-scale-lib-1.1.1.jar /opt/cdh-5.3.6/hive-0.13.1-cdh5.3.6/lib/high-scale-lib-1.1.1.jar

    2、配置hive-site.xml文件,加入zookeeper属性:

    hbase.zookeeper.quorum

    3、配置环境变量:

    export HBASE_HOME=/opt/cdh-5.3.6/hbase-0.98.6-cdh5.3.6
    export HADOOP_HOME=/opt/cdh-5.3.6/hadoop-2.5.0-cdh5.3.6
    HADOOP_CLASSPATH=`${HBASE_HOME}/bin/hbase mapredcp`:${HBASE_HOME}/conf

    4、hive集成hbase中dept表(外部表:hbase中已经存在的表,想要在hive中使用sql进行分析在hive应该创建外部表):

    CREATE EXTERNAL TABLE hbase_dept(
    deptno string,
    dname string,
    loc string)
    STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
    WITH SERDEPROPERTIES ("hbase.columns.mapping" = ":key,info:dname,info:loc")
    TBLPROPERTIES ("hbase.table.name" = "dept");

  • 相关阅读:
    Oracle EBS R12 (12.1.3) Installation Linux(64 bit)
    LINUX的磁盘管理du命令详解
    Solution for Error FRM-92095: Oracle Jnitiator version too low
    EBS learning history (to know about the environment)
    MD5 Checksums for R12.1.1 Rapid Install Media (文档 ID 802195.1)
    adpatch options=hotpatch
    B树和B+树
    Elasticsearch倒排索引
    mysql主从复制
    Found option without preceding group in config file /etc/mysql/my.cnf
  • 原文地址:https://www.cnblogs.com/wcwen1990/p/7643978.html
Copyright © 2011-2022 走看看