blinksql 通过维表关联redis 实现hash结构的数据关联

最新推荐文章于 2023-10-29 13:29:10 发布

wppwpp1

最新推荐文章于 2023-10-29 13:29:10 发布

阅读量256

点赞数

分类专栏： flink 文章标签： flink sql java

本文链接：https://blog.csdn.net/wppwpp1/article/details/117621681

版权

flink 专栏收录该内容

69 篇文章 4 订阅

订阅专栏

1、相关sql如下：


CREATE TABLE source_table (
  --必须和Kafka源表中的5个字段的顺序保持一致。
  messageKey VARBINARY,
  `message` VARBINARY,
  `topic` VARCHAR,
  `partition` INT,
  `offset` BIGINT
) WITH (
  `type` = 'kafka011',
  `topic` = 'flink005',
  `group.id` = 'kafka_test_wpp',
  `bootstrap.servers` = 'xxxx:9092,xxxx:9092,xxxx:9092'
 -- `startupMode` = 'TIMESTAMP'
);


create view source_table1 as 
SELECT 
cast(`message` as VARCHAR ) msg 
from source_table;


create table redis_dim (
    id VARCHAR,
    val VARCHAR,
    PRIMARY KEY (id), 
    PERIOD FOR SYSTEM_TIME
  ) with (
  type = 'redis',
  host = 'xxxx.redis.rds.aliyuncs.com',
  port = '6379',
  dbNum = '0',
  password = 'xxxx',
  hashName='hkeyvalue'
);


create table sink_print2(
    msg varchar,
    redisval VARCHAR
)with(
    type='print'
);


insert into sink_print2 
SELECT 
e.msg, 
cast(w.val as VARCHAR ) as redisval
from source_table1 as e left join redis_dim FOR SYSTEM_TIME AS OF PROCTIME() AS w
on e.msg = w.id
;

2、kafka的数据如下：