zoukankan      html  css  js  c++  java
  • HIVE json格式数据的处理

    今天要处理一个以json格式存储的数据,想要直接把json的各个项的数据存入HIVE表中。

    HIVE直接读入json的函数有两个:

    (1)get_json_object(string json_string, string path)

    返回值: string  

    说明:解析json的字符串json_string,返回path指定的内容。如果输入的json字符串无效,那么返回NULL。  

    举例:  

    hive> select  get_json_object(‘{“store”:{“fruit”:[{"weight":8,"type":"apple"},{"weight":9,"type":"pear"}],  “bicycle”:{“price”:19.95,”color”:”red”}}, “email”:”amy@only_for_json_udf_test.net”,   “owner”:”amy” } ‘,’$.owner’) from dual;  

    结果:amy 

    这个函数每次只能返回一个数据项。

    (2)json_tuple(jsonStr, k1, k2, ...)

    参数为一组键k1,k2……和JSON字符串,返回值的元组。该方法比 get_json_object 高效,因为可以在一次调用中输入多个键

    select a.timestamp, b.*
    from log a lateral view json_tuple(a.appevent, 'eventid', 'eventname') b as f1, f2;

    处理数据样例:
    {"GPS_LAT":39.8965125,"GPS_LONG":116.3493225,"GPS_SPEED":20.9993625,"GPS_STATE":"A","GPS_TIME":"2014-01-02 00:00:16","IMEI":"508597","after_oxygen_sensor":132,"air_condion_state":3,"bdoneNo_after_mileage":0,"bdoneNo_zero_mileage":8044,"db_speed":22,"direction_angle":358.2585,"front_oxygen_sensor":64,"instant_fuel":233,"speed":1210,"torque":33,"total_fuel":0}
    处理HIVE语句:
    create table 2014jrtest as select json_tuple(line,'GPS_LAT','GPS_LONG','GPS_SPEED','GPS_STATE','GPS_TIME','IMEI','after_oxygen_sensor','air_condion_state','bdoneNo_after_mileage','bdoneNo_zero_mileage','db_speed','direction_angle','front_oxygen_sensor','instant_fuel','speed','torque','total_fuel') from 2014test;
  • 相关阅读:
    Spring service本类中方法调用另一个方法事务不生效问题(转载)
    JVM垃圾收集器
    LInkedHashMap实现最近被使用(LRU)缓存
    HTML模板与iframe框架
    Mybatis中常用sql语句
    从零到一: 后端接口文档
    Mysql日期处理
    Java-集合框架与数组的实际应用-组装Json字符串
    Mysql查询之 指定顺序排序
    Eclipse中复制项目后,怎么更改项目名等相关配置?
  • 原文地址:https://www.cnblogs.com/wrong5566/p/7324558.html
Copyright © 2011-2022 走看看