zoukankan      html  css  js  c++  java
  • Flink配置文件

    单机测试时的配置:

    state.backend: filesystem
    # 取消的时候保存检查点
    execution.checkpointing.externalized-checkpoint-retention: RETAIN_ON_CANCELLATION
    # 60s 一次检查点
    execution.checkpointing.interval: 60s
    # 检查点语意
    execution.checkpointing.mode: EXACTLY_ONCE
    
    
    # Directory for checkpoints filesystem, when using any of the default bundled
    # state backends.
    #
    # state.checkpoints.dir: hdfs://namenode-host:port/flink-checkpoints
    state.checkpoints.dir: file:///tmp/flink1.12-checkpoints
    # Default target directory for savepoints, optional.
    #
    # state.savepoints.dir: hdfs://namenode-host:port/flink-savepoints
    state.savepoints.dir: file:///tmp/flink1.12-savepoints
    # Flag to enable/disable incremental checkpoints for backends that
    # support incremental checkpoints (like the RocksDB state backend). 
    #
    # state.backend.incremental: false
    
    # The failover strategy, i.e., how the job computation recovers from task failures.
    # Only restart tasks that may have been affected by the task failure, which typically includes
    # downstream tasks and potentially upstream tasks if their produced data is no longer available for consumption.
    
    jobmanager.execution.failover-strategy: region

    生产配置:

    # state.backend: filesystem
    state.backend: rocksdb
    state.backend.incremental: true
    # 以下目录每个集群节点都必须存在或有权自动创建
    state.backend.rocksdb.localdir: /var/data/flink-rosckdb
    # Directory for checkpoints filesystem, when using any of the default bundled
    # state backends.
    #
    state.checkpoints.dir: hdfs:///user/share/flink1.12/flink-checkpoints
    state.checkpoints.num-retained: 5
    # Default target directory for savepoints, optional.
    #
    state.savepoints.dir: hdfs:///user/share/flink1.12/flink-savepoints
    
    # Flag to enable/disable incremental checkpoints for backends that
    # support incremental checkpoints (like the RocksDB state backend). 
    #
    state.backend.incremental: true
    
    # The failover strategy, i.e., how the job computation recovers from task failures.
    # Only restart tasks that may have been affected by the task failure, which typically includes
    # downstream tasks and potentially upstream tasks if their produced data is no longer available for consumption.
    
    #jobmanager.execution.failover-strategy: region
    restart-strategy: fixed-delay
    restart-strategy.fixed-delay.attempts: 10
    restart-strategy.fixed-delay.delay: 30s
  • 相关阅读:
    有向图中的环DAG
    pyltp安装闭坑指南
    pip安装包到不同的python解释器
    WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connec
    词向量
    基于规则的关系抽取
    NLP(十三)中文分词工具的使用尝试
    NLP(十二)依存句法分析的可视化及图分析
    NLP入门(十一)从文本中提取时间
    NLP入门(十)使用LSTM进行文本情感分析
  • 原文地址:https://www.cnblogs.com/yoyowin/p/14751566.html
Copyright © 2011-2022 走看看