基于Kafka Debezium Confluent实现MySql实时同步到ES

实现目标

        基于MySql的Binlog实现Mysql表实时同步到ES。

实现方案

         1、总体技术方案基于Kafka的Connect技术。具体技术内容，不做介绍，网上有相关文章，本文章主要解决如何配置，因为网上大部分配置都是只能简单运行，只能插入，不能实现数据按照主键进行更新和删除同步的。

         2、基于debezium的MySqlConnector插件，完成Mysql同步到Kafka。 此插件的机制与阿里的canal一样，基于模拟slave读取MySql Binlog。读取数据后通过配置将数据转换，实时同步到Kafka。

         3、基于confluent的elasticsearch.sink插件，将Kafka的数据实时同步到ES。confluent是基于Kafka Connect技术，实现多种数据源格式导入导出的工具平台。本文章只需要下载免费版就可以，主要是拿到里面的kafka-connect-elasticsearch，如果不下载整个confluent，网上能找到kafka-connect-elasticsearch也可以。

具体实现

         1、安装Kafka，见网上文章。

         2、配置connect-standalone.properties，服务有单机模式和集群模式，这里介绍单机模式。此文件在kafka安装目录的config文件夹下。

# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements.  See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License.  You may obtain a copy of the License at
#
#    http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

# These are defaults. This file just demonstrates how to override some settings.
bootstrap.servers=192.188.2.59:9092

# The converters specify the format of data in Kafka and how to translate it into Connect data. Every Connect user will
# need to configure these based on the format they want their data in when loaded from or stored into Kafka
key.c