zoukankan      html  css  js  c++  java
  • Flume监控文件实现断点续传

    1、目录配置,实现不了断点续传功能

    2、针对采集原始文件实现断点续传(taildir source才有这个断点续传的功能),可以采用多文件进行配置,也可以采用文件通配符进行配置要采集的文件(比如:*.log)

    参考:

    al.sources = r1
    a1.channels = c1
    a1.sinks = k1
    
    al.sources.r1.type = TAILDIR
    al.sources.r1.positionFile =/home/xxx/position/taildir_position.json
    al.sources.r1.filegroups.f1 = f1 f2
    al.sources.r1.filegroups.f1=/home/log/log1/*.log
    al.sources.r1.filegroups.f1=/home/log/log2/*.log
    al.sources.r1.headers.f1.headerKey1 = valuel #headers 的value值
    a1.sources.r1.fileHeader=true  #设置文件的绝对路径为 headers的key值
    
    
    al.channels.cl.type =  file
    al.channels.dataDirs = /opt/FLumeclient/datas
    al.channels.checkpointDir =/opt/Flumeclient/checkpoint
    al.channels.maxFileSize = 2146435071
    al.channels.c1.capacity =1000000
    al.channels.cl transaction Capacity =500000
    al.channels.cl.minimumRequiredSpace =524288000
    
    al.sinks.k1.type = logger
    al.sources.r1.channels = c1
    al.sinks.k1.channel = c1
  • 相关阅读:
    poj2886 Who Gets the Most Candies?
    poj1201 Intervals
    poj3109 Inner Vertices
    poj1990 MooFest
    poj3368 Frequent values
    NOIP练习赛题目6
    NOIP练习赛题目5
    NOIP练习赛题目4
    NOIP练习赛题目3
    NOIP练习赛题目2
  • 原文地址:https://www.cnblogs.com/-courage/p/14449814.html
Copyright © 2011-2022 走看看