zoukankan      html  css  js  c++  java
  • hive实现update和delete功能

    如果一个表要实现update和delete功能,该表就必须支持ACID,而支持ACID,就必须满足以下条件:
    1、表的存储格式必须是ORC(STORED AS ORC);
    2、表必须进行分桶(CLUSTERED BY (col_name, col_name, ...)  INTO num_buckets BUCKETS);
    3、Table property中参数transactional必须设定为True(tblproperties('transactional'='true'));
    4、以下配置项必须被设定:
    Hive->配置->类别->高级
    Client端:hive-site.xml 的 Hive 客户端高级配置代码段(安全阀)
    hive.support.concurrency – true
    hive.enforce.bucketing – true
    hive.exec.dynamic.partition.mode – nonstrict  
    hive.txn.manager – org.apache.hadoop.hive.ql.lockmgr.DbTxnManager  
    服务端:hive-site.xml 的 Hive 服务高级配置代码段(安全阀)
    hive.compactor.initiator.on – true
    hive.compactor.worker.threads – 1
    hive.txn.manager – org.apache.hadoop.hive.ql.lockmgr.DbTxnManager

    hive>set hive.txn.manager=org.apache.hadoop.hive.ql.lockmgr.DbTxnManager;

    创建表
    create table t1(id int, name string)
    clustered by (id) into 8 buckets
    stored as orc TBLPROPERTIES ('transactional'='true');

    create table test_trancaction
    (user_id Int,name String)
    clustered by (user_id) into 3 buckets
    stored as orc TBLPROPERTIES ('transactional'='true');

    hive> create table test_insert_test(id int,name string) row format delimited fields TERMINATED BY ','; ---临时表
    hive> LOAD DATA LOCAL INPATH '/data/test.txt' OVERWRITE INTO TABLE test_insert_test;
    hive> select * from test_insert_test;
    OK
    1,jerrick
    2,tom
    3,jerry
    4,lily
    5,hanmei
    6,limlei
    7,lucky

    hive>insert into test_trancaction select * from test_insert_test;
    FAILED: SemanticException [Error 10265]: This command is not allowed on an ACID table merge_data.transactions with a non-ACID transaction manager. Failed

    hive>set hive.txn.manager=org.apache.hadoop.hive.ql.lockmgr.DbTxnManager;
    hive>update test_trancaction set name='ccx' where user_id=2;

  • 相关阅读:
    【C/C++】动态内存分配和链表
    【C/C++】递归算法
    UnicodeMath编码教程
    UnicodeMath数学公式编码_翻译(Unicode Nearly Plain
    浅谈Java反射机制
    lvs--小白博客
    python部署lvs
    python部署galery集群
    python第九章:面向对象--小白博客
    python之yagmail模块--小白博客
  • 原文地址:https://www.cnblogs.com/zsfishman/p/12423042.html
Copyright © 2011-2022 走看看