kafka(官网地址:http://kafka.apache.org)是一款分布式消息发布和订阅的系统
在Flume中的KafkaChannel支持Flume与Kafka整合,可以将Kafka当做channel使用,其主要配置项如下
a1.sources = r1 a1.sinks = k1 a1.channels = c1 # source ...... # sink ...... # channel a1.channels.c1.type=org.apache.flume.channel.kafka.KafkaChannel a1.channels.c1.capacity=10000 a1.channels.c1.transactionCapacity=1000 a1.channels.c1.brokerList=192.168.10.83:9092,192.168.10.84:9092 a1.channels.c1.topic=flume_ulog_channel a1.channels.c1.zookeeperConnect=192.168.10.83:2181 # Bind the source and sink to the channel a1.sources.r1.channels = c1 a1.sinks.k1.channel = c1