zoukankan      html  css  js  c++  java
  • hbase与hive集成:hive读取hbase中数据

    1、创建hbase jar包到hive lib目录软连接

    hive需要jar包:
    hive-hbase-handler-0.13.1-cdh5.3.6.jar
    zookeeper-3.4.5-cdh5.3.6.jar
    guava-12.0.1.jar                  --删除hive下低版本jar包,cp hbase中的到hive/lib/下(即需要注意jar包版本)

    ln -s /opt/cdh-5.3.6/hbase-0.98.6-cdh5.3.6/lib/hbase-server-0.98.6-cdh5.3.6.jar /opt/cdh-5.3.6/hive-0.13.1-cdh5.3.6/lib/hbase-server-0.98.6-cdh5.3.6.jar
    ln -s /opt/cdh-5.3.6/hbase-0.98.6-cdh5.3.6/lib/hbase-client-0.98.6-cdh5.3.6.jar /opt/cdh-5.3.6/hive-0.13.1-cdh5.3.6/lib/hbase-client-0.98.6-cdh5.3.6.jar
    ln -s /opt/cdh-5.3.6/hbase-0.98.6-cdh5.3.6/lib/hbase-protocol-0.98.6-cdh5.3.6.jar /opt/cdh-5.3.6/hive-0.13.1-cdh5.3.6/lib/hbase-protocol-0.98.6-cdh5.3.6.jar
    ln -s /opt/cdh-5.3.6/hbase-0.98.6-cdh5.3.6/lib/hbase-it-0.98.6-cdh5.3.6.jar /opt/cdh-5.3.6/hive-0.13.1-cdh5.3.6/lib/hbase-it-0.98.6-cdh5.3.6.jar
    ln -s /opt/cdh-5.3.6/hbase-0.98.6-cdh5.3.6/lib/htrace-core-2.04.jar /opt/cdh-5.3.6/hive-0.13.1-cdh5.3.6/lib/htrace-core-2.04.jar
    ln -s /opt/cdh-5.3.6/hbase-0.98.6-cdh5.3.6/lib/hbase-hadoop2-compat-0.98.6-cdh5.3.6.jar /opt/cdh-5.3.6/hive-0.13.1-cdh5.3.6/lib/hbase-hadoop2-compat-0.98.6-cdh5.3.6.jar
    ln -s /opt/cdh-5.3.6/hbase-0.98.6-cdh5.3.6/lib/hbase-hadoop-compat-0.98.6-cdh5.3.6.jar /opt/cdh-5.3.6/hive-0.13.1-cdh5.3.6/lib/hbase-hadoop-compat-0.98.6-cdh5.3.6.jar
    ln -s /opt/cdh-5.3.6/hbase-0.98.6-cdh5.3.6/lib/high-scale-lib-1.1.1.jar /opt/cdh-5.3.6/hive-0.13.1-cdh5.3.6/lib/high-scale-lib-1.1.1.jar

    2、配置hive-site.xml文件,加入zookeeper属性:

    hbase.zookeeper.quorum

    3、配置环境变量:

    export HBASE_HOME=/opt/cdh-5.3.6/hbase-0.98.6-cdh5.3.6
    export HADOOP_HOME=/opt/cdh-5.3.6/hadoop-2.5.0-cdh5.3.6
    HADOOP_CLASSPATH=`${HBASE_HOME}/bin/hbase mapredcp`:${HBASE_HOME}/conf

    4、hive集成hbase中dept表(外部表:hbase中已经存在的表,想要在hive中使用sql进行分析在hive应该创建外部表):

    CREATE EXTERNAL TABLE hbase_dept(
    deptno string,
    dname string,
    loc string)
    STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
    WITH SERDEPROPERTIES ("hbase.columns.mapping" = ":key,info:dname,info:loc")
    TBLPROPERTIES ("hbase.table.name" = "dept");

  • 相关阅读:
    telnet模拟http訪问
    network: Android 网络推断(wifi、3G与其它)
    Cocos2d-x学习笔记(19)(TestCpp源代码分析-3)
    Thinkphp编辑器扩展类kindeditor用法
    逛自己的微博,回想以前的那个“我”
    微信生成二维码
    [C++]四种方式求解最大子序列求和问题
    Android 颜色渲染(二) 颜色区域划分原理与实现思路
    Android 颜色渲染(一) 颜色选择器 ColorPickerDialog剖析
    Android 图标上面添加提醒(二)使用开源UI类库 Viewbadger
  • 原文地址:https://www.cnblogs.com/wcwen1990/p/7643978.html
Copyright © 2011-2022 走看看