zoukankan      html  css  js  c++  java
  • flume sink kafka

    agent.sources = s1   
    agent.channels = c1 
    agent.sinks = sk1 
     
    agent.sources.s1.type = netcat 
    agent.sources.s1.bind = localhost 
    agent.sources.s1.port = 5678 
    agent.sources.s1.channels = c1 
    
    agent.sinks.sk1.type = org.apache.flume.sink.kafka.KafkaSink
    agent.sinks.sk1.topic = hellotest
    agent.sinks.sk1.brokerList = localhost:9092
    agent.sinks.sk1.requiredAcks = 1
    agent.sinks.sk1.batchSize = 20
    agent.sinks.sk1.channel = c1
     
    agent.channels.c1.type = memory
    agent.channels.c1.capacity = 1000    
    agent.channels.c1.transactionCapacity = 100
    

     开启zookeeper和kafka,创建一个hellotest的topic

    Last login: Tue Mar 21 10:57:36 on ttys004
    lihudeMacBook-Pro:~ SunAndLi$  telnet  localhost 5678
    Trying ::1...
    telnet: connect to address ::1: Connection refused
    Trying 127.0.0.1...
    Connected to localhost.
    Escape character is '^]'.
    lihuhu 
    OK
    lihuhu is my husband
    OK
    

     查看

    Last login: Tue Mar 21 10:56:50 on ttys003
    lihudeMacBook-Pro:~ SunAndLi$ cd kafka_2.11-0.10.1.0/
    lihudeMacBook-Pro:kafka_2.11-0.10.1.0 SunAndLi$ bin/kafka-console-consumer.sh --zookeeper localhost:2181 --topic hellotest --from-beginning
    Using the ConsoleConsumer with old consumer is deprecated and will be removed in a future major release. Consider using the new consumer by passing [bootstrap-server] instead of [zookeeper].
    lihuhu
    lihuhu is my husband
    
  • 相关阅读:
    常用 SQL Server 规范集锦
    让Git忽略所有obj和bin目录的同步
    Sql server 存储过程基础语法
    nginx 站点代理,负载均衡
    CentOS7.0安装Nginx-1.12.0
    CentOS7安装GNOME可视化界面和如何配置IP地址
    开发工具资料集合
    NOIP2018总结反思
    NOIP2018考试报告
    STL基础用法
  • 原文地址:https://www.cnblogs.com/sunyaxue/p/6593992.html
Copyright © 2011-2022 走看看