zoukankan      html  css  js  c++  java
  • hive集成hbase——FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask错误的解决办法

    问题:FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask

    情景:将hbase内建的宽表映射到hive内,出现报错...

    版本:使用的是hadoop三大版本中的CDH,hive:hive-1.1.0-cdh5.14.2.tar.gz   hbase:hbase-1.2.0-cdh5.14.2.tar.gz

    尝试方案一:

    将hbase 的客户端jar 拷入hive lib目录下,再配置hive/conf下的hive-site.xml文件

    详情见https://blog.csdn.net/u012957549/article/details/86562120

    实用方案二:

    将集群中hadoop/etc/hadoop/core-site.xml和hadoop/etc/hadoop/hdfs-site.xml软连接到hbase/conf下(目前cdh版本基本不需要将hbase的客户端jar包导入hive/lib,按自己集群版本而定),代码如下:

    #在hbase/conf下执行该命令
        ln -s /opt/hadoop/hadoop260/etc/hadoop/core-site.xml ./
        ln -s /opt/hadoop/hadoop260/etc/hadoop/hdfs-site.xml ./

    测试代码如下:

    #创建hbase的宽表数据
    create_namespace 'test'
    create 'test:abb',{NAME=>'name'},{NAME=>'age'},{NAME=>'price'}
    
    put 'test:abb','1','name:first_name','jack'
    put 'test:abb','1','name:last_name','Li'
    put 'test:abb','1','age:xu_age','21'
    put 'test:abb','1','age:shi_age','22'
    put 'test:abb','1','price:costPrice','1200.00'
    put 'test:abb','1','price:retailPrice','2000.00'
    
    put 'test:abb','2','name:first_name','Milk'
    put 'test:abb','2','name:last_name','WL'
    put 'test:abb','2','age:xu_age','24'
    put 'test:abb','2','age:shi_age','25'
    put 'test:abb','2','price:costPrice','3000.00'
    put 'test:abb','2','price:retailPrice','3400.00'
    
    put 'test:abb','3','name:first_name','Lisa'
    put 'test:abb','3','name:last_name','Jams'
    put 'test:abb','3','age:xu_age','23'
    put 'test:abb','3','age:shi_age','24'
    put 'test:abb','3','price:costPrice','2000.00'
    put 'test:abb','3','price:retailPrice','2500.00'
    
    #hadoop与hbase建立软连接
    ln -s /opt/hadoop/hadoop260/etc/hadoop/core-site.xml  /opt/hbase/conf
    ln -s /opt/hadoop/hadoop260/etc/hadoop/hdfs-site.xml  /opt/hbase/conf
    
    #将hbase表映射到hive
    create  external table abb(
    rowkey STRING,
    name_first_name STRING,
    name_last_name STRING,
    age_xu_age STRING,
    age_shi_age STRING,
    price_costPrice STRING,
    price_retailPrice STRING
    )
    stored by 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
    with serdeproperties("hbase.columns.mapping"=":key,name:first_name,name:last_name,age:xu_age,age:shi_age,price:costPrice,price:retailPrice")
    tblproperties("hbase.table.name"="test:abb")
    
    #建立软连接方法亲测有效,具体操作参考自己情况

  • 相关阅读:
    spring常用注解
    P1255 数楼梯
    蓝桥杯 传纸条(动态规划)
    蓝桥杯 数的划分两种解法
    蓝桥杯 数独
    Elasticsearch05-批量增删改查
    Elasticsearch04-正排索引和倒排索引
    Elasticsearch03-Mapping和聚合
    Elasticsearch02-查询语法
    亿级流量多级缓存高并发系统架构实战
  • 原文地址:https://www.cnblogs.com/afeiiii/p/13967232.html
Copyright © 2011-2022 走看看