zoukankan      html  css  js  c++  java
  • flume sink kafka

    agent.sources = s1   
    agent.channels = c1 
    agent.sinks = sk1 
     
    agent.sources.s1.type = netcat 
    agent.sources.s1.bind = localhost 
    agent.sources.s1.port = 5678 
    agent.sources.s1.channels = c1 
    
    agent.sinks.sk1.type = org.apache.flume.sink.kafka.KafkaSink
    agent.sinks.sk1.topic = hellotest
    agent.sinks.sk1.brokerList = localhost:9092
    agent.sinks.sk1.requiredAcks = 1
    agent.sinks.sk1.batchSize = 20
    agent.sinks.sk1.channel = c1
     
    agent.channels.c1.type = memory
    agent.channels.c1.capacity = 1000    
    agent.channels.c1.transactionCapacity = 100
    

     开启zookeeper和kafka,创建一个hellotest的topic

    Last login: Tue Mar 21 10:57:36 on ttys004
    lihudeMacBook-Pro:~ SunAndLi$  telnet  localhost 5678
    Trying ::1...
    telnet: connect to address ::1: Connection refused
    Trying 127.0.0.1...
    Connected to localhost.
    Escape character is '^]'.
    lihuhu 
    OK
    lihuhu is my husband
    OK
    

     查看

    Last login: Tue Mar 21 10:56:50 on ttys003
    lihudeMacBook-Pro:~ SunAndLi$ cd kafka_2.11-0.10.1.0/
    lihudeMacBook-Pro:kafka_2.11-0.10.1.0 SunAndLi$ bin/kafka-console-consumer.sh --zookeeper localhost:2181 --topic hellotest --from-beginning
    Using the ConsoleConsumer with old consumer is deprecated and will be removed in a future major release. Consider using the new consumer by passing [bootstrap-server] instead of [zookeeper].
    lihuhu
    lihuhu is my husband
    
  • 相关阅读:
    jekins构建自动化项目的步骤
    CRT 和mysql 中文乱码解决方式
    Jenkins的配置(rpm red hat方式)
    MapReduce job.setNumReduceTasks(0)思考
    浏览器angent分析工具
    npm中的 --save-dev
    computed与methods的异同
    JS函数种类详解
    Vue.js和Nodejs的关系
    AJAX复习笔记
  • 原文地址:https://www.cnblogs.com/sunyaxue/p/6593992.html
Copyright © 2011-2022 走看看