zoukankan      html  css  js  c++  java
  • hbase与hive集成:hive读取hbase中数据

    1、创建hbase jar包到hive lib目录软连接

    hive需要jar包:
    hive-hbase-handler-0.13.1-cdh5.3.6.jar
    zookeeper-3.4.5-cdh5.3.6.jar
    guava-12.0.1.jar                  --删除hive下低版本jar包,cp hbase中的到hive/lib/下(即需要注意jar包版本)

    ln -s /opt/cdh-5.3.6/hbase-0.98.6-cdh5.3.6/lib/hbase-server-0.98.6-cdh5.3.6.jar /opt/cdh-5.3.6/hive-0.13.1-cdh5.3.6/lib/hbase-server-0.98.6-cdh5.3.6.jar
    ln -s /opt/cdh-5.3.6/hbase-0.98.6-cdh5.3.6/lib/hbase-client-0.98.6-cdh5.3.6.jar /opt/cdh-5.3.6/hive-0.13.1-cdh5.3.6/lib/hbase-client-0.98.6-cdh5.3.6.jar
    ln -s /opt/cdh-5.3.6/hbase-0.98.6-cdh5.3.6/lib/hbase-protocol-0.98.6-cdh5.3.6.jar /opt/cdh-5.3.6/hive-0.13.1-cdh5.3.6/lib/hbase-protocol-0.98.6-cdh5.3.6.jar
    ln -s /opt/cdh-5.3.6/hbase-0.98.6-cdh5.3.6/lib/hbase-it-0.98.6-cdh5.3.6.jar /opt/cdh-5.3.6/hive-0.13.1-cdh5.3.6/lib/hbase-it-0.98.6-cdh5.3.6.jar
    ln -s /opt/cdh-5.3.6/hbase-0.98.6-cdh5.3.6/lib/htrace-core-2.04.jar /opt/cdh-5.3.6/hive-0.13.1-cdh5.3.6/lib/htrace-core-2.04.jar
    ln -s /opt/cdh-5.3.6/hbase-0.98.6-cdh5.3.6/lib/hbase-hadoop2-compat-0.98.6-cdh5.3.6.jar /opt/cdh-5.3.6/hive-0.13.1-cdh5.3.6/lib/hbase-hadoop2-compat-0.98.6-cdh5.3.6.jar
    ln -s /opt/cdh-5.3.6/hbase-0.98.6-cdh5.3.6/lib/hbase-hadoop-compat-0.98.6-cdh5.3.6.jar /opt/cdh-5.3.6/hive-0.13.1-cdh5.3.6/lib/hbase-hadoop-compat-0.98.6-cdh5.3.6.jar
    ln -s /opt/cdh-5.3.6/hbase-0.98.6-cdh5.3.6/lib/high-scale-lib-1.1.1.jar /opt/cdh-5.3.6/hive-0.13.1-cdh5.3.6/lib/high-scale-lib-1.1.1.jar

    2、配置hive-site.xml文件,加入zookeeper属性:

    hbase.zookeeper.quorum

    3、配置环境变量:

    export HBASE_HOME=/opt/cdh-5.3.6/hbase-0.98.6-cdh5.3.6
    export HADOOP_HOME=/opt/cdh-5.3.6/hadoop-2.5.0-cdh5.3.6
    HADOOP_CLASSPATH=`${HBASE_HOME}/bin/hbase mapredcp`:${HBASE_HOME}/conf

    4、hive集成hbase中dept表(外部表:hbase中已经存在的表,想要在hive中使用sql进行分析在hive应该创建外部表):

    CREATE EXTERNAL TABLE hbase_dept(
    deptno string,
    dname string,
    loc string)
    STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
    WITH SERDEPROPERTIES ("hbase.columns.mapping" = ":key,info:dname,info:loc")
    TBLPROPERTIES ("hbase.table.name" = "dept");

  • 相关阅读:
    217. 存在重复元素
    189. 旋转数组
    122. 买卖股票的最佳时机 II
    26. 删除排序数组中的重复项
    [剑指Offer]二进制中1的个数
    [Unity]Unity更改黑色主题(个人版)
    [Unity]限制两个物体之间的距离
    [Untiy]贪吃蛇大作战(五)——游戏主界面
    sql server 函数详解(3)数据类型转换函数和文本图像函数
    sql server 函数详解(2)数学函数
  • 原文地址:https://www.cnblogs.com/wcwen1990/p/7643978.html
Copyright © 2011-2022 走看看