kafka-connect-jdbc 增加source报错:query may not be combined with whole-table copying settings

kafka版本:0.10.2.0

kafka-connect-jdbc插件:10.3.3

增加一个MySQL表作为源监听数据变化。

配置:

{
    "name": "mysql-source-table",
    "config": {
        "connector.class": "io.confluent.connect.jdbc.JdbcSourceConnector",
        "tasks.max": 1,
        "connection.url": "jdbc:mysql://localhost:3306/db?user=abc&password=123456&useUnicode=true&characterEncoding=utf8&allowMultiQueries=true&serverTimezone=GMT%2B8",
        "mode": "timestamp",
        "query": "select * from (select * from test_table where status = 2) a ",
        "timestamp.column.name": "datatime",
        "table.whitelist": "test_table",
        "topic.prefix": "mysql-kafka-",
        "validate.non.null": false
    }
}

然后通过webapi post新增,后台报错:

[2022-04-03 11:09:31,439] ERROR Error while starting connector mysql-source-table (org.apache.kafka.connect.runtime.WorkerConnector)
org.apache.kafka.connect.errors.ConnectException: query may not be combined with whole-table copying settings.
        at io.confluent.connect.jdbc.JdbcSourceConnector.start(JdbcSourceConnector.java:108)
        at org.apache.kafka.connect.runtime.WorkerConnector.doStart(WorkerConnector.java:100)
        at org.apache.kafka.connect.runtime.WorkerConnector.start(WorkerConnector.java:125)
        at org.apache.kafka.connect.runtime.WorkerConnector.transitionTo(WorkerConnector.java:182)
        at org.apache.kafka.connect.runtime.Worker.startConnector(Worker.java:183)
        at org.apache.kafka.connect.runtime.distributed.DistributedHerder.startConnector(DistributedHerder.java:876)
        at org.apache.kafka.connect.runtime.distributed.DistributedHerder.access$1200(DistributedHerder.java:101)
        at org.apache.kafka.connect.runtime.distributed.DistributedHerder$15.call(DistributedHerder.java:892)
        at org.apache.kafka.connect.runtime.distributed.DistributedHerder$15.call(DistributedHerder.java:888)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
Sun Apr 03 11:12:07 CST 2022 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.

一顿搜索后没有找到解决办法,最后根据报错提示,去查看源码:

 at io.confluent.connect.jdbc.JdbcSourceConnector.start(JdbcSourceConnector.java:108)

一看才知道,如果配置query参数,就不能同时配置whitelist和blacklist,即表名。

同时,如果whitelist不配置,topic.prefix必须独立配置一个好区分的标识,因为没有表名拼接后,主题就只得一个前缀,有需要区分topic的,就要注意噢。

 

评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值