Flink实战之MysqlCDC To Iceberg

网上找了一圈没找到一个mysql to Iceberg的实践案例，总结下这几天的踩坑心血历程，希望能提供一些帮助。

现状

目前Iceberg 0.11版本只支持CDC streaming写入，sql 写入以及CDC 读取都是不支持的。
mysql binlog的读取有现成的Connector可用，参考 flink-cdc-connectors。不错demo里面只提供了String的序列化方式，Iceberg需要接受RowData的数据格式。

踩坑记录

java.lang.ClassCastException: org.apache.flink.table.data.GenericRowData cannot be cast to org.apache.flink.types.Row

TypeInformation 构造的不对

Client does not support authentication protocol requested by server; consider upgrading MySQL client

解决方法：https://stackoverflow.com/questions/50093144/mysql-8-0-client-does-not-support-authentication-protocol-requested-by-server

2021-03-12 14:07:57
org.apache.kafka.connect.errors.ConnectException: Access denied; you need (at least one of) the RELOAD privilege(s) for this operation Error code: 1227; SQLSTATE: 42000.
	at io.debezium.connector.mysql.AbstractReader.wrap(AbstractReader.java:230)
	at io.debezium.connector.mysql.AbstractReader.failed(AbstractReader.java:207)
	at io.debezium.connector.mysql.SnapshotReader.execute(SnapshotReader.java:831)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.