Flinksql cep demo


-- 创建CATALOG
CREATE CATALOG `FLINK_CATALOG` WITH('type' = 'generic_in_memory');
-- 创建DATABASE
CREATE DATABASE IF NOT EXISTS `FLINK_CATALOG`.`FLINK_DATABASE`;

--   引用数据表:  [源表]
--   type: kafka
--   datasourceName: engine_kafka_70
--   tableName: kcep
CREATE TABLE IF NOT EXISTS `FLINK_CATALOG`.`FLINK_DATABASE`.`kcep` (
    `symbol` STRING
    ,`price` BIGINT
    ,`tax` BIGINT
    ,`rowtime` TIMESTAMP(3)
    ,WATERMARK FOR rowtime AS rowtime
) WITH (
	 'connector' = 'kafka',
 	 'topic' = 'kcep',
	 'properties.bootstrap.servers' = 'xxxx:9092',
	 'properties.group.id' = '6e02688c-0dc3-40b7-8c9e-643cb87f7a66',
	 'scan.startup.mode' = 'latest-offset',
	'topicId' = '1832974242631647267',
	'format' = 'json'
);



SELECT *
FROM `FLINK_CATALOG`.`FLINK_DATABASE`.`kcep` MATCH_RECOGNIZE (
  ORDER BY rowtime
  MEASURES
    A.price AS aprice,
    B.price AS bprice,
    A.rowtime AS atime,
    B.rowtime AS btime
  PATTERN (A B) WITHIN INTERVAL '2' MINUTES
  DEFINE
    A AS symbol = '男',
    B AS symbol = '女'
) AS T

示例数据格式: 

message:{"symbol":"男","rowtime":"2024-09-09 11:20:24","price":1,"tax":2}

示例结果格式: 

参阅:

Flink动态CEP快速入门_实时计算 Flink版(Flink)-阿里云帮助中心 (aliyun.com)

如何使用复杂事件处理(CEP)语句_实时计算 Flink版(Flink)-阿里云帮助中心 (aliyun.com)

Flink CEP(Complex Event Processing)是一种基于流数据的复杂事件处理框架,能够对流数据中的复杂事件进行识别和处理。Flink CEP SQL 是指在 Flink SQL 中使用 Flink CEP 的语法和函数来进行复杂事件处理。 Flink CEP SQL 需要用到 Flink CEP 库,可以通过以下方式引入: ``` <dependency> <groupId>org.apache.flink</groupId> <artifactId>flink-cep_2.11</artifactId> <version>${flink.version}</version> </dependency> ``` 使用 Flink CEP SQL 可以通过以下步骤来进行: 1. 定义事件流:使用 Flink SQL 定义输入事件流。 ``` CREATE TABLE input_events ( id BIGINT, name STRING, time TIMESTAMP(3), temperature DOUBLE ) WITH ( 'connector.type' = 'kafka', 'connector.version' = 'universal', 'connector.topic' = 'input_topic', 'connector.startup-mode' = 'earliest-offset', 'connector.properties.zookeeper.connect' = 'localhost:2181', 'connector.properties.bootstrap.servers' = 'localhost:9092', 'format.type' = 'json', 'format.derive-schema' = 'true' ); ``` 2. 定义模式:使用 Flink CEP 的语法和函数定义要匹配的模式。 ``` SELECT * FROM input_events MATCH_RECOGNIZE ( PARTITION BY id ORDER BY time MEASURES FIRST(A.temperature) AS start_temperature, LAST(A.temperature) AS end_temperature ONE ROW PER MATCH PATTERN (A B) DEFINE A AS A.temperature < 10.0, B AS B.temperature > 20.0 ) ``` 3. 输出结果:使用 Flink SQL 输出处理后的结果。 ``` CREATE TABLE output_events ( id BIGINT, start_temperature DOUBLE, end_temperature DOUBLE ) WITH ( 'connector.type' = 'kafka', 'connector.version' = 'universal', 'connector.topic' = 'output_topic', 'connector.properties.zookeeper.connect' = 'localhost:2181', 'connector.properties.bootstrap.servers' = 'localhost:9092', 'format.type' = 'json', 'format.derive-schema' = 'true' ); INSERT INTO output_events SELECT id, start_temperature, end_temperature FROM ( SELECT * FROM input_events MATCH_RECOGNIZE ( PARTITION BY id ORDER BY time MEASURES FIRST(A.temperature) AS start_temperature, LAST(A.temperature) AS end_temperature ONE ROW PER MATCH PATTERN (A B) DEFINE A AS A.temperature < 10.0, B AS B.temperature > 20.0 ) ) AS result; ``` 在上述例子中,我们定义了一个模式,用来匹配温度在某一时间段内低于 10 度,并在接下来的时间段内高于 20 度的事件序列。然后我们将匹配的结果输出到 output_events 表中。 总体来说,Flink CEP SQL 提供了一种简单、易用、高效的方式来进行复杂事件处理,可以让用户通过 SQL 语句来定义和处理复杂事件。同时,Flink CEP SQL 也能够和 Flink 的其他功能结合使用,如流处理、批处理等。
评论 1
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值