zoukankan      html  css  js  c++  java
  • Flume 1.4.0 User Guide

    Apache Flume is a distributed, reliable, and available system for efficiently collecting, aggregating and moving large amounts of log data from many different sources to a centralized data store.

    A Flume event is defined as a unit of data flow having a byte payload and an optional set of string attributes. A Flume agent is a (JVM) process that hosts the components through which events flow from an external source to the next destination (hop).

    Agent component diagram

    An agent is started using a shell script called flume-ng which is located in the bin directory of the Flume distribution. You need to specify the agent name, the config directory, and the config file on the command line:

    $ bin/flume-ng agent -n $agent_name -c conf -f conf/flume-conf.properties.template
    

    Here, we give an example configuration file, describing a single-node Flume deployment. This configuration lets a user generate events and subsequently logs them to the console.

    # example.conf: A single-node Flume configuration
    
    # Name the components on this agent
    a1.sources = r1
    a1.sinks = k1
    a1.channels = c1
    
    # Describe/configure the source
    a1.sources.r1.type = netcat
    a1.sources.r1.bind = localhost
    a1.sources.r1.port = 44444
    
    # Describe the sink
    a1.sinks.k1.type = logger
    
    # Use a channel which buffers events in memory
    a1.channels.c1.type = memory
    a1.channels.c1.capacity = 1000
    a1.channels.c1.transactionCapacity = 100
    
    # Bind the source and sink to the channel
    a1.sources.r1.channels = c1
    a1.sinks.k1.channel = c1
    


     

  • 相关阅读:
    Eureka与ZooKeeper 的比较(转)
    springcloud 与分布式系统(转载)
    jvm 分代回收算法通俗理解
    关于JVM调优
    Java常用的数据结构
    mysql数据库引擎(转载)
    java的堆栈通俗理解
    ArrayList 的实现原理
    hashMap 源码解读理解实现原理和hash冲突
    Sklearn库例子——决策树分类
  • 原文地址:https://www.cnblogs.com/aukle/p/3233628.html
Copyright © 2011-2022 走看看