环境:
CentOS7.6
CDH6.3.1
Flink1.4.0
PYTHON3.7.12
其中,Flink已经集成到CDH6.3.1中,如需获取集成过程,请关注本博客。
简介:
从一个例子出发,模拟一遍checkpoint的设置,基于checkpoint的任务的运行时,及模拟job出现异常后从checkpoint恢复的过程。
以下为python脚本(checkpoint_test.py):
import logging
import sys
from pyflink.table import (EnvironmentSettings, TableEnvironment, DataTypes)
from pyflink.datastream import StreamExecutionEnvironment, TimeCharacteristic, CheckpointingMode, ExternalizedCheckpointCleanup, RocksDBStateBackend, FsStateBackend, MemoryStateBackend
from pyflink.table import StreamTableEnvironment, EnvironmentSettings, TableConfig, DataTypes, CsvTableSink, WriteMode, SqlDialect
from pyflink.table.udf import ScalarFunction, udf, udaf
from pyflink.table import AggregateFunction, DataTypes, ListView
from