flink入门

1.安装flink，启动flink

文档地址：Apache Flink 1.3-SNAPSHOT 中文文档: Apache Flink 中文文档

代码：GitHub - apache/flink: Apache Flink

jar下载      Index of /dist/flink  

1。开启mysql binlog

2. flink代码参考（flink1.16.0+flinkcdc 2.3.0）问题1已经采用方案1解决

3.执行sql

4. 开放端口： 8081（flink 默认端口），直接解压linux-flink-1.16.0.rar启动即可

下载地址：百度网盘微信

注意点： 执行多个job配置：参考下面问题2

5.关于运行job方法

 方法1.  打成jar, 上传到flink服务器

 

方法二：main方法运行

问题1. flink启动出现如下问题：

Caused by: java.lang.NoSuchMethodError: io.debezium.connector.mysql.MySqlConnection$MySqlConnectionConfiguration.<init>(Lio/debezium/config/Configuration;Ljava/util/Properties;)V
    at com.ververica.cdc.connectors.mysql.debezium.DebeziumUtils.createMySqlConnection(DebeziumUtils.java:85)
erica.cdc.connectors.mysql.source.reader.MySqlSplitReader.getSnapshotSplitRea

方案1，访问地址：

[Bug] update flink version to 1.16.2and CDC version to 2.3.0:NoSuchMethodError · Issue #2340 · ververica/flink-cdc-connectors · GitHub

方案2. 访问fix review · ververica/flink-cdc-connectors@5415bcf · GitHub

把flink-cdc-connectors 源码修改（源码地址：GitHub - ververica/flink-cdc-connectors: CDC Connectors for Apache Flink®）

2.1.flink-connector-mysql-cdc/src/main/java/com/ververica/cdc/connectors/mysql/MySqlValidator.java

2.2.flink-connector-mysql-cdc/src/main/java/com/ververica/cdc/connectors/mysql/debezium/DebeziumUtils.java

2.3 flink-connector-mysql-cdc/src/main/java/io/debezium/connector/mysql/MySqlConnection.java

 * Copyright Debezium Authors.
 *
 * Licensed under the Apache Software License version 2.0, available at http://www.apache.org/licenses/LICENSE-2.0
 */

package io.debezium.connector.mysql;

import io.debezium.DebeziumException;
import io.debezium.config.CommonConnectorConfig;
import io.debezium.config.CommonConnectorConfig.EventProcessingFailureHandlingMode;
import io.debezium.config.Configuration;
import io.debezium.config.Configuration.Builder;
import io.debezium.config.Field;
import io.debezium.connector.mysql.MySqlConnectorConfig.SecureConnectionMode;
import io.debezium.connector.mysql.legacy.MySqlJdbcContext.DatabaseLocales;
import io.debezium.jdbc.JdbcConnection;
import io.debezium.relational.Column;
import io.debezium.relational.Table;
import io.debezium.relational.TableId;
import io.debezium.relational.history.DatabaseHistory;
import io.debezium.schema.DatabaseSchema;
import io.debezium.util.Strings;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

import java.sql.Connection;
import java.sql.ResultSet;
import java.sql.SQLException;
import java.time.Duration;
import java.util.ArrayList;
import java.util.Collections;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.OptionalLong;
import java.util.Properties;

/**
 * Copied from Debezium project(1.6.4.final) to add custom jdbc properties in the jdbc url. The new
 * parameter {@code jdbcProperties} in the constructor of {@link MySqlConnectionConfiguration} will
 * be used to generate the jdbc url pattern, and may overwrite the default value.
 *
 * <p>Line 80: Add field {@code urlPattern} in {@link MySqlConnection} and remove old pattern.
 *
 * <p>Line 93: Init {@code urlPattern} using the url pattern from {@link
 * MySqlConnectionConfiguration}.
 *
 * <p>Line 561: Generate the connection string by the new field {@code urlPattern}.
 *
 * <p>Line 574 ~ 581: Add new constant and field {@code urlPattern} to {@link
 * MySqlConnectionConfiguration}.
 *
 * <p>Line 626 ~ 629: Init new field {@code urlPattern} in {@link MySqlConnectionConfiguration}.
 *
 * <p>Line 690 ~ 720: Add some methods helping to generate the url pattern and add default values.
 */
public class MySqlConnection extends JdbcConnection {

    private static Logger LOGGER = LoggerFactory.getLogger(MySqlConnection.class);

    private static final String SQL_SHOW_SYSTEM_VARIABLES = "SHOW VARIABLES";
    private static final String SQL_SHOW_SYSTEM_VARIABLES_CHARACTER_SET =
            "SHOW VARIABLES WHERE Variable_name IN ('character_set_server','collation_server')";
    private static final String SQL_SHOW_SESSION_VARIABLE_SSL_VERSION =
            "SHOW SESSION STATUS LIKE 'Ssl_version'";

    private final Map<String, String> originalSystemProperties = new HashMap<>();
    private final MySqlConnectionConfiguration connectionConfig;
    private final MysqlFieldReader mysqlFieldReader;
    private final String urlPattern;

    /**
     * Creates a new connection using the supplied configuration.
     *
     * @param connectionConfig {@link MySqlConnectionConfiguration} instance, may not be null.
     * @param fieldReader binary or text protocol based readers
     */
    public MySqlConnection(
            MySqlConnectionConfiguration connectionConfig, MysqlFieldReader fieldReader) {
        super(connectionConfig.config(), connectionC