zoukankan      html  css  js  c++  java
  • flink Periodic Watermarks 自定义周期性水印

    1、BoundedOutOfOrdernessGenerator

    /**
     * This generator generates watermarks assuming that elements arrive out of order,
     * but only to a certain degree. The latest elements for a certain timestamp t will arrive
     * at most n milliseconds after the earliest elements for timestamp t.
     */
    public class BoundedOutOfOrdernessGenerator implements AssignerWithPeriodicWatermarks<MyEvent> {
    
        private final long maxOutOfOrderness = 3000; // 3.0 seconds
    
        private long currentMaxTimestamp;
    
        @Override
        public long extractTimestamp(MyEvent element, long previousElementTimestamp) {
            long timestamp = element.getCreationTime();
            currentMaxTimestamp = Math.max(timestamp, currentMaxTimestamp);
            return timestamp;
        }
    
        @Override
        public Watermark getCurrentWatermark() {
            // return the watermark as current highest timestamp minus the out-of-orderness bound
            return new Watermark(currentMaxTimestamp - maxOutOfOrderness);
        }
    }

    效果解析:

    2、TimeLagWatermarkGenerator

    /**
     * This generator generates watermarks that are lagging behind processing time by a fixed amount.
     * It assumes that elements arrive in Flink after a bounded delay.
     */
    public class TimeLagWatermarkGenerator implements AssignerWithPeriodicWatermarks<MyEvent> {
    
        private final long maxTimeLag = 3000; // 3 seconds
    
        @Override
        public long extractTimestamp(MyEvent element, long previousElementTimestamp) {
            return element.getCreationTime();
        }
    
        @Override
        public Watermark getCurrentWatermark() {
            // return the watermark as current time minus the maximum time lag
            return new Watermark(System.currentTimeMillis() - maxTimeLag);
        }
    }

    效果解析:

     
  • 相关阅读:
    mac PHP安装imageMagic扩展
    使用AWS Lambda,API Gateway和S3 Storage快速调整图片大小
    composer Changed current directory to没反应
    mongodb批量update更新数据
    php 判断图片文件的真实类型
    C#选择文件、选择文件夹、打开文件(或者文件夹)
    C#连接数据库
    湿寒阳虚体质如何艾灸
    女人艾灸穴位
    四个穴位掌控全身健康
  • 原文地址:https://www.cnblogs.com/asker009/p/11318290.html
Copyright © 2011-2022 走看看