07 友盟项目--拆分日志为五个表---动态生成日志类对应的建日志表的sql 语句-创建表

拆分日志为五个表---动态生成日志类对应的建日志表的sql 语句

1.util包下 ---》 LogUtil  通过内省的方式的到成员变量 

    /**
     * 生成日志类对应的日志表sql语句
     */
    public static <T extends  AppBaseLog> String genDDL(Class<T> clazz) throws Exception {
        String classname = clazz.getSimpleName().toLowerCase() ;
        StringBuffer buffer = new StringBuffer() ;
        buffer.append("create table if not exists ") ;
        buffer.append(classname + "s (
") ;
        //内省 --》 JavaBean
        BeanInfo bi = Introspector.getBeanInfo(clazz) ;
        PropertyDescriptor[] pps = bi.getPropertyDescriptors() ;
        for(int i =0 ; i < pps.length ; i ++ ){
            String name = pps[i].getName();
            Method get = pps[i].getReadMethod() ;
            Method set = pps[i].getWriteMethod();
            if(get != null && set != null){
                if(name.equalsIgnoreCase("createdAtMs")){
                    buffer.append("  createdatms bigint") ;
                }
                else{
                    buffer.append("  " + name.toLowerCase() + " string") ;
                }
                //最后一个
                if(i != (pps.length - 1)){
                    buffer.append(" ,
");
                }
            }
        }
        buffer.append(")
");
        buffer.append("PARTITIONED BY (ym string, day string) 
");
        buffer.append("stored as parquet ;
");
        return buffer.toString() ;
    }

2.测试并生成五个类对应的5个表的建表sql语句

代码

@Test
public void testGenDDL() throws Exception {
   System.out.println(LogUtil.genDDL(AppStartupLog.class));
   System.out.println(LogUtil.genDDL(AppEventLog.class));
   System.out.println(LogUtil.genDDL(AppErrorLog.class));
   System.out.println(LogUtil.genDDL(AppUsageLog.class));
   System.out.println(LogUtil.genDDL(AppPageLog.class));
}

3.生成的建表sql语句如下

create table if not exists appstartuplogs (
  appchannel string ,
  appid string ,
  appplatform string ,
  appversion string ,
  brand string ,
  carrier string ,
  country string ,
  createdatms bigint ,
  deviceid string ,
  devicestyle string ,
  ipaddress string ,
  network string ,
  ostype string ,
  province string ,
  screensize string ,
  tenantid string)
PARTITIONED BY (ym string, day string)
stored as parquet ;

create table if not exists appeventlogs (
  appchannel string ,
  appid string ,
  appplatform string ,
  appversion string ,
  brand string ,
  createdatms bigint ,
  deviceid string ,
  devicestyle string ,
  eventdurationsecs string ,
  eventid string ,
  ostype string ,
  paramkeyvaluemap string ,
  tenantid string)
PARTITIONED BY (ym string, day string)
stored as parquet ;

create table if not exists apperrorlogs (
  appchannel string ,
  appid string ,
  appplatform string ,
  appversion string ,
  brand string ,
  createdatms bigint ,
  deviceid string ,
  devicestyle string ,
  errorbrief string ,
  errordetail string ,
  ostype string ,
  tenantid string)
PARTITIONED BY (ym string, day string)
stored as parquet ;

create table if not exists appusagelogs (
  appchannel string ,
  appid string ,
  appplatform string ,
  appversion string ,
  brand string ,
  createdatms bigint ,
  deviceid string ,
  devicestyle string ,
  ostype string ,
  singledownloadtraffic string ,
  singleuploadtraffic string ,
  singleusedurationsecs string ,
  tenantid string)
PARTITIONED BY (ym string, day string)
stored as parquet ;

create table if not exists apppagelogs (
  appchannel string ,
  appid string ,
  appplatform string ,
  appversion string ,
  brand string ,
  createdatms bigint ,
  deviceid string ,
  devicestyle string ,
  nextpage string ,
  ostype string ,
  pageid string ,
  pageviewcntinsession string ,
  staydurationsecs string ,
  tenantid string ,
  visitindex string)
PARTITIONED BY (ym string, day string)
stored as parquet ;

 注意

上面建表语句  appeventlogs中有错误   删除paramKeyValueMap  字段

4. sql脚本
nano create_logs.sql

5。执行脚本

hive -f create_logs_table.sql