flinkSql join redis的hash结构维表简单实现

最新推荐文章于 2024-08-06 21:16:31 发布

wppwpp1

最新推荐文章于 2024-08-06 21:16:31 发布

阅读量842

点赞数

分类专栏： flink UDF 文章标签： redis kafka big data

本文链接：https://blog.csdn.net/wppwpp1/article/details/109864244

版权

flink 同时被 2 个专栏收录

69 篇文章 4 订阅

订阅专栏

UDF

5 篇文章 0 订阅

订阅专栏

以下是阿里云的blinksql的demo

CREATE TABLE source_table (
  --必须和Kafka源表中的5个字段的顺序保持一致。
  messageKey VARBINARY,
  `message` VARBINARY,
  `topic` VARCHAR,
  `partition` INT,
  `offset` BIGINT
) WITH (
  `type` = 'kafka011',
  `topic` = 'kafka_test',
  `group.id` = 'kafka_test_wpp',
  `bootstrap.servers` = 'xx.xx.xx:9092'
 -- `startupMode` = 'TIMESTAMP'
);



create table redis_dim (
    id VARCHAR,
    name VARCHAR,
    PRIMARY KEY (id), 
    PERIOD FOR SYSTEM_TIME
  ) with (
  type = 'redis',
  host = '127.0.0.1',
  port = '6379',
  dbNum = '0',
  password = 'pass',
  hashName='hashtest'
);


create table sink_print2(
    mes varchar,
    name VARCHAR
)with(
    type='print'
);



insert into sink_print2 
SELECT 
cast (e.`message` as varchar),
 w.name
from source_table as e left join redis_dim FOR SYSTEM_TIME AS OF PROCTIME() AS w
on cast(e.message as varchar )  = w.id
;

维表join相关测试案例

flinksql正则表达式

REGEXP_EXTRACT(`acm`, 'cid_\\d+', 0) AS regex_cid2,

REGEXP_EXTRACT(`acm`, '(cid_\\d+)', 1) AS regex_cid3,