Flink CDC 3.1.0 pipeline 多表合一、分库分表 Mysql写入Doris yaml

SunTecTec

已于 2025-04-27 14:45:51 修改

阅读量454

点赞数 3

分类专栏： Flink 文章标签： flink 大数据

于 2024-06-20 14:04:17 首次发布

本文链接：https://blog.csdn.net/m0_63628018/article/details/139831026

版权

Flink 专栏收录该内容

52 篇文章

订阅专栏

多表合一 yaml：

source:
  type: mysql
  hostname: 172.xxx.xxx.xxx
  port: 3306
  username: root
  password: xxx
  tables: sync_db.\.*
  server-id: 5400-5404
  server-time-zone: Asia/Shanghai

sink:
  type: doris
  fenodes: 172.xxx.xxx.xxx:8030
  username: root
  password: xxx
  table.create.properties.light_schema_change: true

route:
  - source-table: sync_db.\.*
    sink-table: sync_db.ods_sync_db

pipeline:
  name: Sync MySQL Database to Doris
  parallelism: 2

分库分表 yaml：

source:
  type: mysql
  hostname: 172.xxx.xxx.xxx
  port: 3306
  username: root
  password: xxx
  tables: sync_db.\.*
  server-id: 5400-5404
  server-time-zone: Asia/Shanghai

sink:
  type: doris
  fenodes: 172.xxx.xxx.xxx:8030
  username: root
  password: xxx
  table.create.properties.light_schema_change: true

route:
  - source-table: sync_db.teacher
    sink-table: syc.ods_teacher
  - source-table: sync_db.student
    sink-table: syc.ods_student

pipeline:
  name: Sync MySQL Database to Doris
  parallelism: 2