zoukankan      html  css  js  c++  java
  • flume发送与接受日志

    apache-flume-1.5.0.1-bin.tar.gz
    
    日志接收端
    
    1.修改配置文件
    cp flume-conf.properties.template flume-conf.properties
    vim flume-conf.properties
    collector1.sources = AvroIn
    collector1.sources.AvroIn.type = avro
    collector1.sources.AvroIn.bind = 0.0.0.0
    collector1.sources.AvroIn.port = 4545
    collector1.sources.AvroIn.channels = mc1
    
    collector1.channels = mc1
    collector1.channels.mc1.type = memory
    collector1.channels.mc1.capacity = 100
    
    collector1.sinks = LocalOut
    
    collector1.sinks.LocalOut.type = file_roll
    collector1.sinks.LocalOut.sink.directory = /var/log/flume/collector1
    collector1.sinks.LocalOut.sink.rollInterval = 0
    collector1.sinks.LocalOut.channel = mc1
    
    
    cp flume-env.sh.template  flume-env.sh
    vim flume-env.sh
    修改JAVA_HOME
    
    
    2.运行
    bin/flume-ng agent -c conf -f conf/flume-conf.properties -n collector1
    
    日志发送端
    1.修改配置文件
    cp flume-conf.properties.template flume-conf.properties
    vim flume-conf.properties
    agent1.sources = ngrinder
    agent1.sources.ngrinder.type = exec 
    agent1.sources.ngrinder.command = tail -F /var/log/apache2/access.log
    agent1.sources.ngrinder.channels = mc1
    
    agent1.channels = mc1
    agent1.channels.mc1.type = memory
    agent1.channels.mc1.capacity = 100
    
    agent1.sinks = avro-sink
    
    agent1.sinks.avro-sink.type = avro
    agent1.sinks.avro-sink.channel = mc1
    agent1.sinks.avro-sink.hostname = 192.168.0.111
    agent1.sinks.avro-sink.port = 4545
    #缓存日志断点续传
    agent1.channels.memory4log.type = memory 
    agent1.channels.memory4log.capacity = 1000 
    agent1.channels.memory4log.transactionCapacity = 100
    
    
    cp flume-env.sh.template  flume-env.sh
    vim flume-env.sh
    修改JAVA_HOME
    
    2.执行
    bin/flume-ng agent -c conf -f conf/flume-conf.properties -n agent1
    
    3.注意
    /var/log/flume/collector1/ 要写入权限
    /var/log/apache2/access.log  要读取权限
  • 相关阅读:
    webpack的入门实践,看这篇就够了
    vue系列教程-15vuex的使用
    vue系列教程-14axios的使用
    vue系列教程-13vuecli初体验
    vue系列教程-12vue单文件组件开发
    vue系列教程-11vuerouter路由
    vue系列教程-10vue过滤器和自定义指令
    vue系列教程-09vue组件
    Spring Redis开启事务支持错误用法导致服务不可用
    MySQL主从数据库配置与原理
  • 原文地址:https://www.cnblogs.com/DDSys/p/4027369.html
Copyright © 2011-2022 走看看