现象:
运营一段时间点maxwell发现数据抽取异常,检查maxwell运行日志,发现了如下错误日志。
报错日志:
2022-01-01 23:18:17,799 [shutdownThread] INFO TaskManager - Stopping: com.zendesk.maxwell.schema.PositionStoreThread@14b39e8f
2022-01-01 23:18:17,799 [shutdownThread] INFO StoppableTaskState - com.zendesk.maxwell.schema.PositionStoreThread requestStop() called (in state: RUNNING)
2022-01-01 23:18:17,799 [shutdownThread] INFO TaskManager - Stopping: com.zendesk.maxwell.producer.MaxwellKafkaProducerWorker@1a747269
2022-01-01 23:18:17,799 [shutdownThread] INFO StoppableTaskState - MaxwellKafkaProducerWorker requestStop() called (in state: RUNNING)
2022-01-01 23:18:17,801 [shutdownThread] INFO KafkaProducer - [Producer clientId=producer-1] Closing the Kafka producer with timeoutMillis = 9223372036854775807 ms.
2022-01-01 23:18:17,803 [shutdownThread] INFO TaskManager - Stopping: com.zendesk.maxwell.bootstrap.BootstrapController@2bfdf1d8
2022-01-01 23:18:17,804 [shutdownThread] INFO StoppableTaskState - com.zendesk.maxwell.bootstrap.BootstrapController requestStop() called (in state: RUNNING)
2022-01-01 23:18:17,804 [shutdownThread] INFO TaskManager - Stopping: com.zendesk.maxwell.replication.BinlogConnectorReplicator@3b66871d
2022-01-01 23:18:17,804 [shutdownThread] INFO StoppableTaskState - com.zendesk.maxwell.replication.BinlogConnectorReplicator requestStop() called (in state: STOPPED)
com.github.shyiko.mysql.binlog.network.ServerException: Could not find first log file name in binary log index file
at com.github.shyiko.mysql.binlog.BinaryLogClient.listenForEventPackets(BinaryLogClient.java:946)
at com.github.shyiko.mysql.binlog.BinaryLogClient.connect(BinaryLogClient.java:581)
at com.github.shyiko.mysql.binlog.BinaryLogClient$7.run(BinaryLogClient.java:857)
at java.lang.Thread.run(Thread.java:745)
错误原因:
MySQL主从库不同步,比如断点、不优雅的重启了主、从进程。
解决方案:
# 方案 1:
mysql> truncate table `positions`;
Query OK, 0 rows affected (0.16 sec)
# 方案 2:
① 停止所有的maxwell进程
② mysql> drop database maxwell;
Query OK, 7 rows affected (1.26 sec)
③ 重启所有的maxwell进程。
# 方案3(针对MySQL 主从库同步而言):
(1) 首先停止从库同步:
mysql >slave stop;
(2) 主库中关闭当前的二进制日志文件并创建一个新文件,新的二进制日志文件的名字在当前的二进制文件的编号上加1.
mysql >flush logs;
(3) 查看主库状态,主要查看日志文件和位置:
mysql >show master status;
(4) 回到从库中,执行命令,使日志文件和位置对应主库:
mysql >CHANGE MASTER TO MASTER_LOG_FILE='log-bin.000005',MASTER_LOG_POS=107;
(5) 最后,启动从库:
mysql >slave start;
mysql >show slave status;
状态如下,基本上是正常了,可以主库修改,测试一下从库是否同步。
Slave_IO_State: Waiting for master to send event
Slave_IO_Running: Yes
Slave_SQL_Running: Yes