zoukankan      html  css  js  c++  java
  • flume与kafka整合

    flume与kafka整合

    前提:##

    在上诉条件满足的情况下才能进行flume和kafka的整合。

    flume与kafka整合

    • 修改/usr/local/flume/conf/flume-conf.properties
    agent.sinks.s1.type = org.apache.flume.sink.kafka.KafkaSink
    agent.sinks.s1.topic = mytopic
    agent.sinks.s1.brokerList = localhost:9092
    agent.sinks.s1.requiredAcks = 1
    agent.sinks.s1.batchSize = 20
    agent.sinks.s1.channel = c1
    

    启动kafka

    • 启动zookeeper

    /usr/local/kafka/bin/zookeeper-server-start.sh -daemon config/zookeeper.properties

    • 启动kafka

    /usr/local/kafka/bin/kafka-server-start.sh config/server.properties

    • 创建主题test

    /usr/local/kafka/bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic test

    启动flume

    /usr/local/flume/bin/flume-ng agent --conf conf -f conf/flume-conf.properties -n agent&

    验证

    • 新开一个客户端

    telnet localhost 8888

    [root@localhost ~]# telnet localhost 8888
    Trying ::1...
    telnet: connect to address ::1: Connection refused
    Trying 127.0.0.1...
    Connected to localhost.
    Escape character is '^]'.
    test
    OK
    chat
    OK
    this is a flume message to kafka
    OK
    from flume message.
    OK
    it's flume and kafka message。
    OK
    
    
    • kafka查看消息接收情况
    [root@localhost kafka]# /usr/local/kafka/bin/kafka-console-consumer.sh --zookeeper localhost:2181 --topic test --from-beginning
    test
    chat
    this is a flume message to kafka
    from flume message.
    it's flume and kafka message。
    
    

    如果出现上述效果则说明flume和kafka整合成功。

  • 相关阅读:
    多步操作产生错误,请检查每一步的状态值
    MediaPlayer 播放百度歌曲
    MusicPlayer
    wpf slider 控件模板
    c# 静态构造函数
    好吧,学了久c#,params都不知道怎么用,记录下
    C# 使用各种API
    WPF转义字符
    WPF 执行完一段动画后再关闭窗口
    WPF xml的绑定
  • 原文地址:https://www.cnblogs.com/rwxwsblog/p/5800385.html
Copyright © 2011-2022 走看看