MySQL主从同步报错故障处理总结:
一、主从同步报错故障处理1
Error_code: 1032; handler error HA_ERR_KEY_NOT_FOUND
原因是由于, 主库要同步更新到从库, 而从库没有这条记录, 导致 1032 错误, 修复的话就是找回 从库 丢失的 记录
利用mysqlbinlog工具找出42303的事件
/usr/local/mysql-5.6.30/bin/mysqlbinlog --base64-output=decode-rows -vv mysql-bin.012769 |grep -A 20 '184060304'
或者
/usr/local/mysql-5.6.30/bin/mysqlbinlog --base64-output=decode-rows -vv mysql-bin.012769 --stop-position=184060304 | tail -20
或
usr/local/mysql-5.6.30/bin/mysqlbinlog --base64-output=decode-rows -vv mysql-bin.012769 > decode.log
( 或者加上参数-d, –database=name 来进一步过滤)
方法二:更新从库
mysql> insert into app_svip_member values (1,'****',****);
Query OK, 1 row affected (0.01 sec)
mysql> start slave;
Query OK, 0 rows affected (0.03 sec)
mysql> show slave status\G;
Slave_IO_Running: Yes
Slave_SQL_Running: Yes
方法三:保证生产能正常使用,先跳过1032
stop slave ; set global sql_slave_skip_counter=1; start slave;
show slave status\G;
二、主从同步报错故障处理2
mysql-5.6.20主从同步错误之Error_code: 1032; handler error HA_ERR_KEY_NOT_FOUND
方法一:
1.Error_code: 1032; handler error HA_ERR_KEY_NOT_FOUND 是主从更新时丢失数据,导致主从不一致,在从库上mysql> show slave status\G;结果如下
2.在master上,用mysqlbinlog 分析下出错的binlog日志在干什么:
/usr/local/mysql/bin/mysqlbinlog --no-defaults -v -v --base64-output=DECODE-ROWS mysql-bin.000042 | grep -A '10' 455223179
3.查询master和slave上对应的数据,主库有而从库没有
select * from log_silver where id=9097514\G;
4.把丢失的数据在slave上填补,然后跳过报错
mysql> insert into t1 values();
mysql> stop slave ;set global sql_slave_skip_counter=1;start slave;
5.查看同步结果
mysql> show slave status\G;
方法二:
1.如果用方法一还是不能主从同步,查询到这个错误是mysql的bug导致的
2.解决的办法:
1.最好的办法是升级数据库 保证bug不会重现。
2.利用配置参数 来躲避这个bug vi /etc/my.cnf
slave-skip-errors = 1032,xxxx,xxxx ....
3.临时逃避此次错误。
set global sql_slave_skip_counter=1; stop slave; start slave;
三、主从同步报错故障处理3
在发生故障切换后,经常遇到的问题就是同步报错,下面是最近收集的报错信息。
记录删除失败
在master上删除一条记录,而slave上找不到
Last_SQL_Error: Could not execute Delete_rows event on table hcy.t1;
Can't find record in 't1',
Error_code: 1032; handler error HA_ERR_KEY_NOT_FOUND;
the event's master log mysql-bin.000006, end_log_pos 254
解决方法:master要删除一条记录,而slave上找不到报错,这种情况主都已经删除了,那么从机可以直接跳过。
stop slave;
set global sql_slave_skip_counter=1;
start slave;
如果这种情况很多,需要针对这种错误专门写相关脚本。
主键重复
在slave已经有该记录,又在master上插入了同一条记录。
Last_SQL_Error: Could not execute Write_rows event on table hcy.t1;
Duplicate entry '2' for key 'PRIMARY',
Error_code: 1062;
handler error HA_ERR_FOUND_DUPP_KEY; the event's master log mysql-bin.000006, end_log_pos 924
解决方法:
在slave上用desc hcy.t1; 先看下表结构:
mysql> desc hcy.t1;
+-------+---------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+-------+---------+------+-----+---------+-------+
| id | int(11) | NO | PRI | 0 | |
| name | char(4) | YES | | NULL | |
+-------+---------+------+-----+---------+-------+
删除重复的主键
mysql> delete from t1 where id=2;
Query OK, 1 row affected (0.00 sec)
mysql> start slave;
Query OK, 0 rows affected (0.00 sec)
mysql> show slave status\G;
……
Slave_IO_Running: Yes
Slave_SQL_Running: Yes
……
mysql> select * from t1 where id=2;
在master上和slave上再分别确认一下。
更新丢失
在master上更新一条记录,而slave上找不到,丢失了数据。
Last_SQL_Error: Could not execute Update_rows event on table hcy.t1;
Can't find record in 't1',
Error_code: 1032;
handler error HA_ERR_KEY_NOT_FOUND;
the event's master log mysql-bin.000010, end_log_pos 794
解决方法:
在master上,用mysqlbinlog 分析下出错的binlog日志在干什么。
/usr/local/mysql/bin/mysqlbinlog --no-defaults -v -v --base64-output=DECODE-ROWS mysql-bin.000010 | grep -A '10' 794
#120302 12:08:36 server id 22 end_log_pos 794 Update_rows: table id 33 flags: STMT_END_F
### UPDATE hcy.t1
### WHERE
### @1=2 /* INT meta=0 nullable=0 is_null=0 */
### @2='bbc' /* STRING(4) meta=65028 nullable=1 is_null=0 */
### SET
### @1=2 /* INT meta=0 nullable=0 is_null=0 */
### @2='BTV' /* STRING(4) meta=65028 nullable=1 is_null=0 */
# at 794
#120302 12:08:36 server id 22 end_log_pos 821 Xid = 60
COMMIT/*!*/;
DELIMITER ;
# End of log file
ROLLBACK /* added by mysqlbinlog */;
/*!50003 SET COMPLETION_TYPE=@OLD_COMPLETION_TYPE*/;
在slave上,查找下更新后的那条记录,应该是不存在的。
mysql> select * from t1 where id=2;
Empty set (0.00 sec)
然后再到master查看
mysql> select * from t1 where id=2;
+----+------+
| id | name |
+----+------+
| 2 | BTV |
+----+------+
1 row in set (0.00 sec)
把丢失的数据在slave上填补,然后跳过报错即可。
mysql> insert into t1 values (2,'BTV');
Query OK, 1 row affected (0.00 sec)
mysql> select * from t1 where id=2;
+----+------+
| id | name |
+----+------+
| 2 | BTV |
+----+------+
1 row in set (0.00 sec)
mysql> stop slave ;set global sql_slave_skip_counter=1;start slave;
Query OK, 0 rows affected (0.01 sec)
Query OK, 0 rows affected (0.00 sec)
Query OK, 0 rows affected (0.00 sec)
mysql> show slave status\G;
……
Slave_IO_Running: Yes
Slave_SQL_Running: Yes
……
1236错误, 二进制文件缺失
误删二进制文件等各种原因,导致主库mysql-bin.000012文件丢失,从库同步失败。
Master_Log_File: mysql-bin.000012
Slave_IO_Running: No
Slave_SQL_Running: Yes
Last_IO_Error: Got fatal error 1236 from master when reading data from binary log: 'Could not find first log file name in binary log index file'
- 首先停止从库同步
slave stop;
- 查看主库日志文件和位置
mysql> show master logs;
+------------------+-----------+
| Log_name | File_size |
+------------------+-----------+
| mysql-bin.000013 | 154 |
+------------------+-----------+
- 回从库,使日志文件和位置对应主库
CHANGE MASTER TO MASTER_LOG_FILE='log-bin.000013',MASTER_LOG_POS=154;
- 最后,启动从库:
slave start;
show slave status\G;
Master_Log_File: mysql-bin.000013
Slave_IO_Running: Yes
Slave_SQL_Running: Yes
Last_IO_Error:
中继日志损坏
slave的中继日志relay-bin损坏。
Last_SQL_Error: Error initializing relay log position: I/O error reading the header from the binary log
Last_SQL_Error: Error initializing relay log position: Binlog has bad magic number;
It's not a binary log file that can be used by this version of MySQL
1、手工修复
解决方法:找到同步的binlog和POS点,然后重新做同步,这样就可以有新的中继日值了。
例子:
mysql> show slave status\G;
*************************** 1. row ***************************
Master_Log_File: mysql-bin.000010
Read_Master_Log_Pos: 1191
Relay_Log_File: vm02-relay-bin.000005
Relay_Log_Pos: 253
Relay_Master_Log_File: mysql-bin.000010
Slave_IO_Running: Yes
Slave_SQL_Running: No
Replicate_Do_DB:
Replicate_Ignore_DB:
Replicate_Do_Table:
Replicate_Ignore_Table:
Replicate_Wild_Do_Table:
Replicate_Wild_Ignore_Table:
Last_Errno: 1593
Last_Error: Error initializing relay log position: I/O error reading the header from the binary log
Skip_Counter: 1
Exec_Master_Log_Pos: 821
Slave_IO_Running :接收master的binlog信息
Master_Log_File
Read_Master_Log_Pos
Slave_SQL_Running:执行写操作
Relay_Master_Log_File
Exec_Master_Log_Pos
以执行写的binlog和POS点为准。
Relay_Master_Log_File: mysql-bin.000010
Exec_Master_Log_Pos: 821
mysql> stop slave;
Query OK, 0 rows affected (0.01 sec)
mysql> CHANGE MASTER TO MASTER_LOG_FILE='mysql-bin.000010',MASTER_LOG_POS=821;
Query OK, 0 rows affected (0.01 sec)
mysql> start slave;
Query OK, 0 rows affected (0.00 sec)
mysql> show slave status\G;
*************************** 1. row ***************************
Slave_IO_State: Waiting for master to send event
Master_Host: 192.168.8.22
Master_User: repl
Master_Port: 3306
Connect_Retry: 10
Master_Log_File: mysql-bin.000010
Read_Master_Log_Pos: 1191
Relay_Log_File: vm02-relay-bin.000002
Relay_Log_Pos: 623
Relay_Master_Log_File: mysql-bin.000010
Slave_IO_Running: Yes
Slave_SQL_Running: Yes
Replicate_Do_DB:
Replicate_Ignore_DB:
Replicate_Do_Table:
Replicate_Ignore_Table:
Replicate_Wild_Do_Table:
Replicate_Wild_Ignore_Table:
Last_Errno: 0
Last_Error:
Skip_Counter: 0
Exec_Master_Log_Pos: 1191
Relay_Log_Space: 778
Until_Condition: None
Until_Log_File:
Until_Log_Pos: 0
Master_SSL_Allowed: No
Master_SSL_CA_File:
Master_SSL_CA_Path:
Master_SSL_Cert:
Master_SSL_Cipher:
Master_SSL_Key:
Seconds_Behind_Master: 0
Master_SSL_Verify_Server_Cert: No
Last_IO_Errno: 0
Last_IO_Error:
Last_SQL_Errno: 0
Last_SQL_Error:
2、Ibbackup
各种大招都用上了,无奈slave数据丢失过多,ibbackup(需要银子)该你登场了。
Ibbackup热备份工具,是付费的。xtrabackup是免费的,功能上一样。
Ibbackup备份期间不锁表,备份时开启一个事务(相当于做一个快照),然后会记录一个点,之后数据的更改保存在ibbackup_logfile文件里,恢复时把ibbackup_logfile 变化的数据再写入到ibdata里。
Ibbackup 只备份数据( ibdata、.ibd ),表结构.frm不备份。