zoukankan      html  css  js  c++  java
  • flume发送与接受日志

    apache-flume-1.5.0.1-bin.tar.gz
    
    日志接收端
    
    1.修改配置文件
    cp flume-conf.properties.template flume-conf.properties
    vim flume-conf.properties
    collector1.sources = AvroIn
    collector1.sources.AvroIn.type = avro
    collector1.sources.AvroIn.bind = 0.0.0.0
    collector1.sources.AvroIn.port = 4545
    collector1.sources.AvroIn.channels = mc1
    
    collector1.channels = mc1
    collector1.channels.mc1.type = memory
    collector1.channels.mc1.capacity = 100
    
    collector1.sinks = LocalOut
    
    collector1.sinks.LocalOut.type = file_roll
    collector1.sinks.LocalOut.sink.directory = /var/log/flume/collector1
    collector1.sinks.LocalOut.sink.rollInterval = 0
    collector1.sinks.LocalOut.channel = mc1
    
    
    cp flume-env.sh.template  flume-env.sh
    vim flume-env.sh
    修改JAVA_HOME
    
    
    2.运行
    bin/flume-ng agent -c conf -f conf/flume-conf.properties -n collector1
    
    日志发送端
    1.修改配置文件
    cp flume-conf.properties.template flume-conf.properties
    vim flume-conf.properties
    agent1.sources = ngrinder
    agent1.sources.ngrinder.type = exec 
    agent1.sources.ngrinder.command = tail -F /var/log/apache2/access.log
    agent1.sources.ngrinder.channels = mc1
    
    agent1.channels = mc1
    agent1.channels.mc1.type = memory
    agent1.channels.mc1.capacity = 100
    
    agent1.sinks = avro-sink
    
    agent1.sinks.avro-sink.type = avro
    agent1.sinks.avro-sink.channel = mc1
    agent1.sinks.avro-sink.hostname = 192.168.0.111
    agent1.sinks.avro-sink.port = 4545
    #缓存日志断点续传
    agent1.channels.memory4log.type = memory 
    agent1.channels.memory4log.capacity = 1000 
    agent1.channels.memory4log.transactionCapacity = 100
    
    
    cp flume-env.sh.template  flume-env.sh
    vim flume-env.sh
    修改JAVA_HOME
    
    2.执行
    bin/flume-ng agent -c conf -f conf/flume-conf.properties -n agent1
    
    3.注意
    /var/log/flume/collector1/ 要写入权限
    /var/log/apache2/access.log  要读取权限
  • 相关阅读:
    django channels 模型图
    React-Router 非Route监管组件如何监听location变化?
    djoser
    Integration of hot-load django dev sever and hot-load reactjs dev server
    type annotation / static checker / dynamic checker of python
    Prototype of Draw and Guess Game
    Django Channels -- more than HTTP
    DRF Views Inheritance Relationships
    Django REST framework -- ViewSets & Routers
    Django REST framework -- Relationships & Hyperlinked APIs
  • 原文地址:https://www.cnblogs.com/DDSys/p/4027369.html
Copyright © 2011-2022 走看看