zoukankan      html  css  js  c++  java
  • java.lang.IllegalStateException: File name has been re-used with different files. (flume报错)

    报错日志:

     java.lang.IllegalStateException: File name has been re-used with different files. Spooling assumptions violated for /usr/local/data/flumedata/words.txt.COMPLETED
                at org.apache.flume.client.avro.ReliableSpoolingFileEventReader.rollCurrentFile(ReliableSpoolingFileEventReader.java:378)
                at org.apache.flume.client.avro.ReliableSpoolingFileEventReader.retireCurrentFile(ReliableSpoolingFileEventReader.java:330)
                at org.apache.flume.client.avro.ReliableSpoolingFileEventReader.readEvents(ReliableSpoolingFileEventReader.java:259)
                at org.apache.flume.source.SpoolDirectorySource$SpoolDirectoryRunnable.run(SpoolDirectorySource.java:228)
                at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
                at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
                at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
                at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
                at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
                at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
                at java.lang.Thread.run(Thread.java:745)

    原因:
    重复使用了同一个数据文件
    解决方案1:
    更改flume的配置文件conf的目录
    a1.sources.r1.spoolDir = /usr/local/data/flumedata

    #push mode
    a1.sources = r1
    a1.sinks = k1
    a1.channels = c1
    #source
    a1.sources.r1.channels = c1
    a1.sources.r1.type = spooldir
    a1.sources.r1.spoolDir = /usr/local/data/flumedata
    a1.sources.r1.fileHeader = true
    #channel
    a1.channels.c1.type =memory
    a1.channels.c1.capacity = 20000
    a1.channels.c1.transactionCapacity=5000
    #sinks
    a1.sinks.k1.channel = c1
    a1.sinks.k1.type = avro
    a1.sinks.k1.hostname=192.168.13.24
    a1.sinks.k1.port = 9999
    a1.sinks.k1.batchSize= 2000  

    解决方案2:
    在flume配置文件的目录下重新定义数据文件

  • 相关阅读:
    CSS 中的字体兼容写法:用CSS为英文和中文字体分别设置不同的字体
    利用vue-cropper做的关于图片裁剪、压缩、上传、预览等做的一个公共组件
    解决浏览器拦截弹出窗口问题
    详解Vue中的nextTick
    vue里ref ($refs)用法
    vue组件的hover事件模拟、给第三方组件绑定事件不生效问题
    JS实现千分位
    JS实现异步编程的4种方法
    Cas_个人理解
    zabbix_监控_邮件预警
  • 原文地址:https://www.cnblogs.com/wyh-study/p/12093832.html
Copyright © 2011-2022 走看看