zoukankan      html  css  js  c++  java
  • flume:netcat to hive

    a1.sources=r1
    a1.channels=c1
    a1.sinks=s1
     
    a1.sources.r1.type=netcat
    a1.sources.r1.bind=master
    a1.sources.r1.port=44444
     
    a1.sinks.s1.type=hive
    a1.sinks.s1.hive.metastore=thrift://node1:9083
    a1.sinks.s1.hive.database=bd14
    a1.sinks.s1.hive.table=flume_user
    a1.sinks.s1.serializer=DELIMITED
    a1.sinks.s1.serializer.delimiter="	"
    a1.sinks.s1.serializer.serdeSeparator='	'
    a1.sinks.s1.serializer.fieldnames=user_id,user_name,age
     
    a1.channels.c1.type=memory
    a1.channels.c1.capacity=1000
    a1.channels.c1.transactionCapacity=100
     
    a1.sources.r1.channels=c1
    a1.sinks.s1.channel=c1




    export HIVE_HOME
    =/usr/hdp/current/hive flume-ng agent -c conf -f netcat2hive.conf --name a1




    create table flume_user( user_id
    int ,user_name string ,age int ) clustered by (user_id) into 2 buckets stored as orc tblproperties("transactional"='true');



    telnet localhost
    44444 1 tab键 王五 tab键 15 tab键
  • 相关阅读:
    [BJOI2006]狼抓兔子
    [HNOI2016]最小公倍数
    hihocoder 1419 重复旋律4
    [NOI2015]品酒大会
    [SDOI2016]生成魔咒
    [ZJOI2009]狼和羊的故事
    BZOJ4361 isn
    [SDOI2009]虔诚的墓主人
    BZOJ 3329 Xorequ
    [ZJOI2013]丽洁体
  • 原文地址:https://www.cnblogs.com/felixzh/p/14182932.html
Copyright © 2011-2022 走看看