Flink_Over Windows

Over Windows
• Over window 聚合是标准 SQL 中已有的（over 子句），可以在查询的
SELECT 子句中定义
• Over window 聚合，会针对每个输入行，计算相邻行范围内的聚合
• Over windows 使用 window（w:overwindows*）子句定义，并在 select（）
方法中通过别名来引用
val table = input
.window([w: OverWindow] as 'w)
.select('a, 'b.sum over 'w, 'c.min over 'w)
• Table API 提供了 Over 类，来配置 Over 窗口的属性

Flowing暂时无用 只能开窗到当前行

无界 Over Windows
• 可以在事件时间或处理时间，以及指定为时间间隔、或行计数的范围内，定
义 Over windows
• 无界的 over window 是使用常量指定的
// 无界的事件时间 over window
.window(Over partitionBy 'a orderBy 'rowtime preceding UNBOUNDED_RANGE as 'w)
//无界的处理时间 over window
.window(Over partitionBy 'a orderBy 'proctime preceding UNBOUNDED_RANGE as 'w)
// 无界的事件时间 Row-count over window
.window(Over partitionBy 'a orderBy 'rowtime preceding UNBOUNDED_ROW as 'w)
//无界的处理时间 Row-count over window
.window(Over partitionBy 'a orderBy 'proctime preceding UNBOUNDED_ROW as 'w)

有界 Over Windows
• 有界的 over window 是用间隔的大小指定的
// 有界的事件时间 over window
.window(Over partitionBy 'a orderBy 'rowtime preceding 1.minutes as 'w)
// 有界的处理时间 over window
.window(Over partitionBy 'a orderBy 'proctime preceding 1.minutes as 'w)
// 有界的事件时间 Row-count over window
.window(Over partitionBy 'a orderBy 'rowtime preceding 10.rows as 'w)
// 有界的处理时间 Row-count over window
.window(Over partitionBy 'a orderBy 'proctime preceding 10.rows as 'w)

SQL 中的 Group Windows
• Group Windows 定义在 SQL 查询的 Group By 子句中
Ø TUMBLE(time_attr, interval)
• 定义一个滚动窗口，第一个参数是时间字段，第二个参数是窗口长度
Ø HOP(time_attr, interval, interval)
• 定义一个滑动窗口，第一个参数是时间字段，第二个参数是窗口滑动步长，第三个是
窗口长度
Ø SESSION(time_attr, interval)
• 定义一个会话窗口，第一个参数是时间字段，第二个参数是窗口间隔
案例 每十秒统计一次 对于每个传感器按照时间排序 统计向前两行的平均温度 按照事件时间
//Api
package com.atguigu.tableapiandsql

import com.atguigu.sourceandsink.SensorReading
import org.apache.flink.streaming.api.TimeCharacteristic
import org.apache.flink.streaming.api.functions.timestamps.BoundedOutOfOrdernessTimestampExtractor
import org.apache.flink.streaming.api.scala._
import org.apache.flink.streaming.api.windowing.time.Time
import org.apache.flink.table.api.scala._
import org.apache.flink.table.api.{Over, OverWindowedTable, Table, Tumble}
import org.apache.flink.types.Row

object TestOverWindow {
def main(args: Array[String]): Unit = {
val streamEnv: StreamExecutionEnvironment = StreamExecutionEnvironment.getExecutionEnvironment
streamEnv.setStreamTimeCharacteristic(TimeCharacteristic.EventTime)
val tableEnv: StreamTableEnvironment = StreamTableEnvironment.create(streamEnv)//默认blinkplanner
//读取数据
val inputStream: DataStream[String] = streamEnv.readTextFile(“in/sensor.txt”)
val dataStream: DataStream[SensorReading] = inputStream.map(data => {
val strings: Array[String] = data.split(",")
SensorReading(strings(0), strings(1).toLong, strings(2).toDouble)
}).assignTimestampsAndWatermarks(new BoundedOutOfOrdernessTimestampExtractorSensorReading {
override def extractTimestamp(element: SensorReading): Long = element.timestamp * 1000L
})
val sensorTable: Table = tableEnv.fromDataStream(dataStream,'id,'temperature,'timestamp.rowtime as 'ts)
sensorTable.printSchema()
//窗口测试
//1.over window
//2.每十秒统计一次
//Api 对于每个传感器按照时间排序 统计向前两行的平均温度 按照事件时间

val overWindow: OverWindowedTable = sensorTable.window( Over partitionBy 'id orderBy 'ts preceding 2.rows as 'ow)
val resultAPItable: Table = overWindow.select('id,'ts,'id.count over 'ow,'temperature.avg over 'ow)
//先注册表
tableEnv.createTemporaryView("sensorTable3",sensorTable)
val resultSqlTable: Table = tableEnv.sqlQuery(
  """
select
id,
ts,
count(id) over ow,
avg(temperature) over ow
from
sensorTable3
window ow as(
partition by id
order by ts
rows between 2 preceding and current row
)
  """.stripMargin)
resultAPItable.toAppendStream[Row].print("resultApiTable")

resultSqlTable.toAppendStream[Row].print("resultSqlTable")
streamEnv.execute("time and window test")

}
}