zoukankan      html  css  js  c++  java
  • Hive不常用操作记录

    Hive列转行:

    select school,concat_ws('|',collect_set(name)) as name from tmp_01 group by school;

    注意:collect_set 有重复的会去重, collect_list  有重复的不去重

    hive行转列:

    select school,name,name_key

    from tb_tmp_001

    lateral view explode(split(name,',')) num as name_key where school=‘清华大学’;

    注意:遇到特殊字符(“|”,””,”/”,”,”,”’”)无法分割时,需加 “\” 进行转译

    Hive修改字段注释:

    ALTER TABLE table_name CHANGE COLUMN muid muid_new STRING COMMENT '这里是列注释!';

    例如:alter table tb_test change column id id string comment '用户号码';

    Hive修改表注释:

    ALTER TABLE table_name SET TBLPROPERTIES('comment' = '这是表注释!');

    例如:ALTER TABLE tb_tmp_001 SET TBLPROPERTIES('comment' = '临时表!');

    Hive数据导入:

    create table tb_tmp_flink_sink_test(

    id string,

    sensor string,

    )row format delimited

    fields terminated by ' '

    lines terminated by ' '

    stored as textfile ;

    方式一put:

    Hdfs dfs -put /opt/datas/2nd_xa_05_price.csv /user/hive/warehouse/db_lianjia.db/tb_country/month_str=05/city_str=XA

    ALTER TABLE tb_country ADD IF NOT EXISTS PARTITION (month_str="05",city_str="SH")

    方式二load:

    LOAD DATA LOCAL INPATH '/opt/datas/2nd_bj_05_price.csv'  INTO TABLE tb_country PARTITION (month_str="05",city_str="BJ");

    Hive数据导出:

    方式一导入到本地:

     insert overwrite local directory '/home/wyp/wyp'  select * from wyp;

    方式二导入到hdfs,再下载下来:

    insert overwrite directory '/home/wyp/hdfs'  select * from wyp;

    hdfs get /home/wyp/hdfs /tmp/

    hive创建数据库:

    create database mydb1 location '/user/hive/mydb1';

    hive建表语句:

    2:使用MAP字段存储学生考试成绩

    create table stu2(

    id int,

    name string,

    scores  map<string,int>

    )row format delimited

    fields terminated by ' '

    collection items terminated by ','

    map keys terminated by ':'

    lines terminated by ' '

    stored as textfile location '/user/admin/xiaojun/';hive建表语法中的分隔符设定

    row format delimited 分隔符设置开始语句

    fields terminated by:设置字段与字段之间的分隔符

    collection items terminated by:设置一个复杂类型(array,struct)字段的各个item之间的分隔符

    map keys terminated by:设置一个复杂类型(map)字段的key value之间的分隔符

    lines terminated by:设置行与行之间的分隔符

    Hive切换队列的三种方式:

    set mapred.job.queue.name=queue3;

    SET mapreduce.job.queuename=queue3;

    set mapred.queue.names=queue3;

    HIVE 设置任务名称

    set mapred.job.name = my_job_name

    author@nohert
  • 相关阅读:
    centos 编码问题 编码转换 cd到对应目录 执行 中文解压
    centos 编码问题 编码转换 cd到对应目录 执行 中文解压
    centos 编码问题 编码转换 cd到对应目录 执行 中文解压
    Android MVP 十分钟入门!
    Android MVP 十分钟入门!
    Android MVP 十分钟入门!
    Android MVP 十分钟入门!
    mysql备份及恢复
    mysql备份及恢复
    mysql备份及恢复
  • 原文地址:https://www.cnblogs.com/gzgBlog/p/14902128.html
Copyright © 2011-2022 走看看