zoukankan      html  css  js  c++  java
  • Flink--Window apply

    和window的操作类似,只不过操作更加灵活,具体的操作需要在匿名内部类的方法中实现;当有比较复杂的需求时候,可以使用;

    object WindowApply {
      def main(args: Array[String]): Unit = {
        val env = StreamExecutionEnvironment.getExecutionEnvironment
        val textStream: DataStream[String] = env.socketTextStream("localhost" , 9999)
        val flatmapdata: DataStream[String] = textStream.flatMap(x => x.split(","))
        val mapdata: DataStream[(String, Int)] = flatmapdata.map(line => (line,1))
        val keybyStream: KeyedStream[(String, Int), String] = mapdata.keyBy(line => line._1)
        val window: WindowedStream[(String, Int), String, TimeWindow] = keybyStream.timeWindow(Time.of(1 , TimeUnit.SECONDS) ,Time.of(100,TimeUnit.MILLISECONDS))
        val data = window.apply(new WindowFunction[(String, Int) , (String, Int) , String , TimeWindow] {
          override def apply(key: String,
                             window: TimeWindow,
                             input: Iterable[(String, Int)],
                             out: Collector[(String, Int)]): Unit = {
            var output = ""
            var index = 0
            for(in <- input){
                output += "key :" + in._1 + "   value:"+in._2
                index = index + 1
              out.collect(output , index)
              }
          }
        })
        data.print()
        env.execute()
    
      }
    }

    注意,例子中使用的是window,所以对应的匿名内部类是:WindowFunction

    如果使用的是windowAll,则需要使用的内部类是:AllWindowFunction

  • 相关阅读:
    disconf使用小结
    关于spring aop Advisor排序问题
    关于tomcat WEB-INF/lib下类加载顺序
    Netty5客户端源码解析
    巧用命令行工具 redis-cli
    redis学习总结
    聊聊Redis的持久化
    Git管理代码
    RabbitMQ连接池、生产者、消费者实例
    java处理节假日和工作时间的工具类
  • 原文地址:https://www.cnblogs.com/niutao/p/10548586.html
Copyright © 2011-2022 走看看