zoukankan      html  css  js  c++  java
  • FLINK基础(140):DS流与表转换(6) Handling of Changelog Streams(1)简介

    Internally, Flink’s table runtime is a changelog processor. The concepts page describes how dynamic tables and streams relate to each other.

    StreamTableEnvironment offers the following methods to expose these change data capture (CDC) functionalities:

    • fromChangelogStream(DataStream): Interprets a stream of changelog entries as a table. The stream record type must be org.apache.flink.types.Row since its RowKind flag is evaluated during runtime. Event-time and watermarks are not propagated by default. This method expects a changelog containing all kinds of changes (enumerated in org.apache.flink.types.RowKind) as the default ChangelogMode.

    • fromChangelogStream(DataStream, Schema): Allows to define a schema for the DataStream similar to fromDataStream(DataStream, Schema). Otherwise the semantics are equal to fromChangelogStream(DataStream).

    • fromChangelogStream(DataStream, Schema, ChangelogMode): Gives full control about how to interpret a stream as a changelog. The passed ChangelogMode helps the planner to distinguish between insert-onlyupsert, or retract behavior.

    • toChangelogStream(Table): Reverse operation of fromChangelogStream(DataStream). It produces a stream with instances of org.apache.flink.types.Row and sets the RowKind flag for every record at runtime. All kinds of updating tables are supported by this method. If the input table contains a single rowtime column, it will be propagated into a stream record’s timestamp. Watermarks will be propagated as well.

    • toChangelogStream(Table, Schema): Reverse operation of fromChangelogStream(DataStream, Schema). The method can enrich the produced column data types. The planner might insert implicit casts if necessary. It is possible to write out the rowtime as a metadata column.

    • toChangelogStream(Table, Schema, ChangelogMode): Gives full control about how to convert a table to a changelog stream. The passed ChangelogMode helps the planner to distinguish between insert-onlyupsert, or retract behavior.

    From a Table API’s perspective, converting from and to DataStream API is similar to reading from or writing to a virtual table connector that has been defined using a CREATE TABLE DDL in SQL.

    Because fromChangelogStream behaves similar to fromDataStream, we recommend reading the previous section before continuing here.

    This virtual connector also supports reading and writing the rowtime metadata of the stream record.

    The virtual table source implements SupportsSourceWatermark.

    本文来自博客园,作者:秋华,转载请注明原文链接:https://www.cnblogs.com/qiu-hua/p/15204226.html

  • 相关阅读:
    项目构建之maven篇:2.HelloWorld项目构建过程
    求最大子段和的一些算法
    多线程编程(四)--线程同步
    SICP 解题集 — SICP 解题集
    函数式编程很难,这正是你要学习它的原因 | 外刊IT评论网
    haskell,lisp,erlang你们更喜欢哪个?
    欧舒丹 L'Occitane 活力清泉保湿面霜
    段子
    宽带中国战略_百度百科
    brutal是什么意思_brutal在线翻译_英语_读音_用法_例句_海词词典
  • 原文地址:https://www.cnblogs.com/qiu-hua/p/15204226.html
Copyright © 2011-2022 走看看