zoukankan      html  css  js  c++  java
  • RocksDB State Backend Details

    Incremental Checkpoints

    Recovery time of incremental checkpoints may be longer or shorter compared to full checkpoints. If your network bandwidth is the bottleneck, it may take a bit longer to restore from an incremental checkpoint, because it implies fetching more data (more deltas). Restoring from an incremental checkpoint is faster, if the bottleneck is your CPU or IOPs, because restoring from an incremental checkpoint means not re-building the local RocksDB tables from Flink’s canonical key/value snapshot format (used in savepoints and full checkpoints).

    与完全检查点相比,增量检查点的恢复时间可能更长或更短。如果网络带宽是瓶颈,那么从增量检查点恢复可能需要更长的时间,因为这意味着需要提取更多数据(更多增量)。如果瓶颈是CPU或IOPs,那么从增量检查点恢复速度更快,因为从增量检查点恢复意味着不用从Flink的标准键/值快照格式重新构建本地RocksDB表。

    Limitations of the RocksDBStateBackend:

    • As RocksDB’s JNI bridge API is based on byte[], the maximum supported size per key and per value is 2^31 bytes each. IMPORTANT: states that use merge operations in RocksDB (e.g. ListState) can silently accumulate value sizes > 2^31 bytes and will then fail on their next retrieval. This is currently a limitation of RocksDB JNI.

    RocksDBStateBackend限制: 

    由于RocksDB的 JNI bridge API 基于byte[],每个键和每个值支持的最大大小为2^31字节。重要提示:在RocksDB中使用合并操作的状态(例如ListState)可以轻易地积累值大小> 2^31字节,然后在下一次检索时失败。这是目前RocksDB JNI的一个限制。

    未完待续。

    ref:

    https://ci.apache.org/projects/flink/flink-docs-release-1.10/ops/state/state_backends.html#the-rocksdbstatebackend

  • 相关阅读:
    Run
    axios+Qs请求数据转表单格式
    Vue开发电子书app
    vue2.5开发去哪儿了流程
    ES6重度学习 demo实例
    JS 数组, 对象的增查改删(多语法对比)
    格式化时间戳的n种方法
    Vue中你忽略的点
    vscode代码格式化
    分隔符
  • 原文地址:https://www.cnblogs.com/zgq25302111/p/13184978.html
Copyright © 2011-2022 走看看