zoukankan      html  css  js  c++  java
  • Apache Flume

    An Event is a unit of data that flows through a Flume agent. The Event flows from Source to Channel to Sink, and is represented by an implementation of the Event interface. An Event carries a payload (byte array) that is accompanied by an optional set of headers (string attributes). A Flume agent is a process (JVM) that hosts the components that allow Events to flow from an external source to a external destination.

    Source consumes Events having a specific format, and those Events are delivered to the Source by an external source like a web server. For example, an AvroSource can be used to receive Avro Events from clients or from other Flume agents in the flow. When a Source receives an Event, it stores it into one or more Channels. The Channel is a passive store that holds the Event until that Event is consumed by a Sink. One type of Channel available in Flume is the FileChannel which uses the local filesystem as its backing store. A Sink is responsible for removing an Event from the Channel and putting it into an external repository like HDFS (in the case of an HDFSEventSink) or forwarding it to the Source at the next hop of the flow. The Source and Sink within the given agent run asynchronously with the Events staged in theChannel.
     
  • 相关阅读:
    elasticsearch数据迁移
    Leetcode <剑指 Offer 64. 求1+2+…+n>
    Leetcode <27.移除元素>
    Leetcode <7.整数反转>
    Leetcode <1. 两数之和>
    python实现对于告警规则的判断思路
    Python实现密码生成器
    Autojs 打包APP+签名
    Docker 文档整理
    Pycharm 连接Linux远程开发
  • 原文地址:https://www.cnblogs.com/iiiDragon/p/9329906.html
Copyright © 2011-2022 走看看