zoukankan      html  css  js  c++  java
  • Flume监控文件实现断点续传

    1、目录配置,实现不了断点续传功能

    2、针对采集原始文件实现断点续传(taildir source才有这个断点续传的功能),可以采用多文件进行配置,也可以采用文件通配符进行配置要采集的文件(比如:*.log)

    参考:

    al.sources = r1
    a1.channels = c1
    a1.sinks = k1
    
    al.sources.r1.type = TAILDIR
    al.sources.r1.positionFile =/home/xxx/position/taildir_position.json
    al.sources.r1.filegroups.f1 = f1 f2
    al.sources.r1.filegroups.f1=/home/log/log1/*.log
    al.sources.r1.filegroups.f1=/home/log/log2/*.log
    al.sources.r1.headers.f1.headerKey1 = valuel #headers 的value值
    a1.sources.r1.fileHeader=true  #设置文件的绝对路径为 headers的key值
    
    
    al.channels.cl.type =  file
    al.channels.dataDirs = /opt/FLumeclient/datas
    al.channels.checkpointDir =/opt/Flumeclient/checkpoint
    al.channels.maxFileSize = 2146435071
    al.channels.c1.capacity =1000000
    al.channels.cl transaction Capacity =500000
    al.channels.cl.minimumRequiredSpace =524288000
    
    al.sinks.k1.type = logger
    al.sources.r1.channels = c1
    al.sinks.k1.channel = c1
  • 相关阅读:
    OC中extern,static,const的用法
    pod install 报错
    设置Image渲染模式使用TintColor
    VLC 用到的那些 YUV 格式
    base64编码原理
    scp 拷贝文件时报错
    linux 恢复误删文件
    hadoop 集群安装(Cloudera CDH方式)
    记录自己需要读的几本书
    求解最长回文子串
  • 原文地址:https://www.cnblogs.com/-courage/p/14449814.html
Copyright © 2011-2022 走看看