zoukankan      html  css  js  c++  java
  • Flume 1.4.0 User Guide

    Apache Flume is a distributed, reliable, and available system for efficiently collecting, aggregating and moving large amounts of log data from many different sources to a centralized data store.

    A Flume event is defined as a unit of data flow having a byte payload and an optional set of string attributes. A Flume agent is a (JVM) process that hosts the components through which events flow from an external source to the next destination (hop).

    Agent component diagram

    An agent is started using a shell script called flume-ng which is located in the bin directory of the Flume distribution. You need to specify the agent name, the config directory, and the config file on the command line:

    $ bin/flume-ng agent -n $agent_name -c conf -f conf/flume-conf.properties.template
    

    Here, we give an example configuration file, describing a single-node Flume deployment. This configuration lets a user generate events and subsequently logs them to the console.

    # example.conf: A single-node Flume configuration
    
    # Name the components on this agent
    a1.sources = r1
    a1.sinks = k1
    a1.channels = c1
    
    # Describe/configure the source
    a1.sources.r1.type = netcat
    a1.sources.r1.bind = localhost
    a1.sources.r1.port = 44444
    
    # Describe the sink
    a1.sinks.k1.type = logger
    
    # Use a channel which buffers events in memory
    a1.channels.c1.type = memory
    a1.channels.c1.capacity = 1000
    a1.channels.c1.transactionCapacity = 100
    
    # Bind the source and sink to the channel
    a1.sources.r1.channels = c1
    a1.sinks.k1.channel = c1
    


     

  • 相关阅读:
    在 SQL2005 使用行转列或列转行
    JOIN 和 WHERE?简单的问题也有学问。
    完整的获取表结构语句
    经典背景音乐收集
    interface 是什么类型?
    WMI使用集锦
    存储过程+游标,第一回开发使用
    Sql Server 基本语句汇总
    PowerDesigner 设置各项变量参数的路径
    测试
  • 原文地址:https://www.cnblogs.com/aukle/p/3233628.html
Copyright © 2011-2022 走看看