• Flume1.6.0搭建


    下载地址:http://archive.apache.org/dist/flume/

    解压完毕

    切换到安装目录下/usr/local/flume/apache-flume-1.6.0-bin/conf

    1.简单示例

    1.1 netcat模式

    创建demoagent.conf

    # example.conf: A single-node Flume configuration
    
    # Name the components on this agent
    a1.sources = r1
    a1.sinks = k1
    a1.channels = c1
    
    # Describe/configure the source
    a1.sources.r1.type = netcat
    a1.sources.r1.bind = CentOS7One
    a1.sources.r1.port = 44444
    
    # Describe the sink
    a1.sinks.k1.type = logger
    
    # Use a channel which buffers events in memory
    a1.channels.c1.type = memory
    a1.channels.c1.capacity = 1000
    a1.channels.c1.transactionCapacity = 100
    
    # Bind the source and sink to the channel
    a1.sources.r1.channels = c1
    a1.sinks.k1.channel = c1

    测试:Windows用telnet命令测试

    telnet centos7one 44444

    在cmd终端随便输入几个字符

    在Flume端可以看到返回的消息

     1.2 exec模式

    创建demoagent-exec.conf

    # example.conf: A single-node Flume configuration
    
    # Name the components on this agent
    a1.sources = r1
    a1.sinks = k1
    a1.channels = c1
    
    # Describe/configure the source
    a1.sources.r1.type = exec
    a1.sources.r1.command = tail -f /usr/local/flume/test.log
    a1.sources.r1.port = 44444
    a1.sources.r1.channels = c1
    
    # Describe the sink
    a1.sinks.k1.type = logger
    
    # Use a channel which buffers events in memory
    a1.channels.c1.type = memory
    a1.channels.c1.capacity = 1000
    a1.channels.c1.transactionCapacity = 100
    
    # Bind the source and sink to the channel
    a1.sources.r1.channels = c1
    a1.sinks.k1.channel = c1

     克隆一个窗口在命令行输入如下信息

    对test.log持续添加信息

    [root@centos7one flume]# while true
    > do
    > echo message >> test.log 
    > done

     可以看到Flume端,接收到了实时的消息

  • 相关阅读:
    同步/异步/阻塞/非阻塞
    java io异步
    java nio知识点总结
    使用BBED恢复数据文件头
    Codeforces Round #257 (Div. 2) B Jzzhu and Sequences
    Hibernate @Embeddable注解
    2388 Who's in the Middle(简单排序)
    CorePlot学习
    九度OJ 1179 阶乘(模拟)
    NSHashTable 和 NSMapTable学习
  • 原文地址:https://www.cnblogs.com/Java-Starter/p/10794734.html
走看看 - 开发者的网上家园