版本背景
flink:1.14.3
flinkcdc 2.3.0 /2.2.0
现象:
java.lang.NullPointerException: Name is null
at java.lang.Enum.valueOf(Enum.java:236) ~[?:1.8.0_312]
at com.ververica.cdc.connectors.mysql.source.offset.BinlogOffsetKind.valueOf(BinlogOffsetKind.java:26) ~[flink-connector-mysql-cdc-2.3.0.1.jar:2.3.0]
at com.ververica.cdc.connectors.mysql.source.offset.BinlogOffset.getOffsetKind(BinlogOffset.java:136) ~[flink-connector-mysql-cdc-2.3.0.1.jar:2.3.0]
at com.ververica.cdc.connectors.mysql.source.utils.SerializerUtils.readBinlogPosition(SerializerUtils.java:73) ~[flink-connector-mysql-cdc-2.3.0.1.jar:2.3.0]
at com.ververica.cdc.connectors.mysql.source.utils.SerializerUtils.readBinlogPosition(SerializerUtils.java:59) ~[flink-connector-mysql-cdc-2.3.0.1.jar:2.3.0]
at com.ververica.cdc.connectors.mysql.source.split.MySqlSplitSerializer.deserializeSplit(MySqlSplitSerializer.java:153) ~[flink-connector-mysql-cdc-2.3.0.1.jar:2.3.0]
at com.ververica.cdc.connectors.mysql.source.split.MySqlSplitSerializer.deserialize(MySqlSplitSerializer.java:122) ~[flink-connector-mysql-cdc-2.3.0.1.jar:2.3.0]
at com.ververica.cdc.connectors.mysql.source.split.MySqlSplitSerializer.deserialize(MySqlSplitSerializer.java:46) ~[flink-connector-mysql-cdc-2.3.0.1.jar:2.3.0]
at org.apache.flink.core.io.SimpleVersionedSerialization.readVersionAndDeSerialize(SimpleVersionedSerialization.java:165) ~[flink-core-1.13.5.jar:1.13.5]
at org.apache.flink.streaming.api.operators.util.SimpleVersionedListState$DeserializingIterator.next(SimpleVersionedListState.java:138) ~[flink-streaming-java_2.11-1.13.5.jar:1.13.5]
at java.util.Iterator.forEachRemaining(Iterator.java:116) ~[?:1.8.0_312]
at org.apache.flink.util.CollectionUtil.iterableToList(CollectionUtil.java:95) ~[flink-core-1.13.5.jar:1.13.5]
at org.apache.flink.streaming.api.operators.SourceOperator.open(SourceOperator.java:251) ~[flink-streaming-java_2.11-1.13.5.jar:1.13.5]
at org.apache.flink.streaming.runtime.tasks.OperatorChain.initializeStateAndOpenOperators(OperatorChain.java:442) ~[flink-streaming-java_2.11-1.13.5.jar:1.13.5]
at org.apache.flink.streaming.runtime.tasks.StreamTask.restoreGates(StreamTask.java:585) ~[flink-streaming-java_2.11-1.13.5.jar:1.13.5]
at org.apache.flink.streaming.runtime.tasks.StreamTaskActionExecutor$1.call(StreamTaskActionExecutor.java:55) ~[flink-streaming-java_2.11-1.13.5.jar:1.13.5]
at org.apache.flink.streaming.runtime.tasks.StreamTask.executeRestore(StreamTask.java:565) ~[flink-streaming-java_2.11-1.13.5.jar:1.13.5]
at org.apache.flink.streaming.runtime.tasks.StreamTask.runWithCleanUpOnFail(StreamTask.java:650) ~[flink-streaming-java_2.11-1.13.5.jar:1.13.5]
at org.apache.flink.streaming.runtime.tasks.StreamTask.restore(StreamTask.java:540) ~[flink-streaming-java_2.11-1.13.5.jar:1.13.5]
at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:759) ~[flink-runtime_2.11-1.13.5.jar:1.13.5]
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:566) ~[flink-runtime_2.11-1.13.5.jar:1.13.5]
at java.lang.Thread.run(Thread.java:748) ~[?:1.8.0_312]
原因:
k8s环境cdc版本和yarn环境版本不一致,无法正常从savepoint启动
解决方案:
1.将k8s镜像中版本修改为2.2.0
2.在sink端可以保证一致性的情况写,使用指定时间戳消费mysql数据,减小风险