两台RHEL6.3测试服务器上,存在1主3备的MySQL5.6环境,由于很长时间没有使用,今天启动之后,3台slave分别出现了Got fatal error 1236 from master when reading data from binary log: 'Could not find first log file name in binary log index file'问题,显示没有找到binlog日志。
(system@localhost)[(none)]>show slave status\G
*************************** 1. row ***************************
Slave_IO_State:
Master_Host: 192.168.56.2
Master_User: repl
Master_Port: 3306
Connect_Retry: 60
Master_Log_File: mysql.000021
Read_Master_Log_Pos: 862
Relay_Log_File: mysql-relay-bin.000025
Relay_Log_Pos: 4
Relay_Master_Log_File: mysql.000021
Slave_IO_Running: No
Slave_SQL_Running: Yes
Replicate_Do_DB:
Replicate_Ignore_DB:
Replicate_Do_Table:
Replicate_Ignore_Table:
Replicate_Wild_Do_Table:
Replicate_Wild_Ignore_Table:
Last_Errno: 0
Last_Error:
Skip_Counter: 0
Exec_Master_Log_Pos: 862
Relay_Log_Space: 120
Until_Condition: None
Until_Log_File:
Until_Log_Pos: 0
Master_SSL_Allowed: No
Master_SSL_CA_File:
Master_SSL_CA_Path:
Master_SSL_Cert:
Master_SSL_Cipher:
Master_SSL_Key:
Seconds_Behind_Master: NULL
Master_SSL_Verify_Server_Cert: No
Last_IO_Errno: 1236
Last_IO_Error: Got fatal error 1236 from master when reading data from binary log: 'Could not find first log file name in binary log index file'
Last_SQL_Errno: 0
Last_SQL_Error:
Replicate_Ignore_Server_Ids:
Master_Server_Id: 2433306
Master_UUID: b4cf465a-8447-11e5-ac19-080027796e8a
Master_Info_File: /data/mysqldata/3307/data/master.info
SQL_Delay: 0
SQL_Remaining_Delay: NULL
Slave_SQL_Running_State: Slave has read all relay log; waiting for the slave I/O thread to update it
Master_Retry_Count: 86400
Master_Bind:
Last_IO_Error_Timestamp: 160408 12:42:59
Last_SQL_Error_Timestamp:
Master_SSL_Crl:
Master_SSL_Crlpath:
Retrieved_Gtid_Set:
Executed_Gtid_Set:
Auto_Position: 0
1 row in set (0.01 sec)
从上面的信息可以显示,IO thread有问题,SQL thread没有问题。
检查master上的binlog日志:
[root@Mserver1 binlog]# pwd
/data/mysqldata/3306/binlog
[root@Mserver1 binlog]# ll
total 8
-rw-rw---- 1 mysql mysql 120 Apr 8 12:37 mysql.000022
-rw-rw---- 1 mysql mysql 41 Apr 8 12:37 mysql.index
[root@Mserver1 binlog]# more mysql.index
/data/mysqldata/3306/binlog/mysql.000022
[root@Mserver1 binlog]#
发现只有22号binlog,21号binlog已经没有了。
检查参数expire_logs_days:
(system@localhost)[(none)]>show variables like 'expire_logs_days';
+------------------+-------+
| Variable_name | Value |
+------------------+-------+
| expire_logs_days | 90 |
+------------------+-------+
1 row in set (0.00 sec)
说明我有3个多月没有登录这套测试环境了。
下面需要重置复制环境:
stop slave;
show slave status\G
reset slave;
show slave status\G
start slave;
(system@localhost)[(none)]>stop slave;
Query OK, 0 rows affected (0.32 sec)
(system@localhost)[(none)]>show slave status\G
*************************** 1. row ***************************
Slave_IO_State:
Master_Host: 192.168.56.2
Master_User: repl
Master_Port: 3306
Connect_Retry: 60
Master_Log_File: mysql.000021
Read_Master_Log_Pos: 862
Relay_Log_File: mysql-relay-bin.000025
Relay_Log_Pos: 4
Relay_Master_Log_File: mysql.000021
Slave_IO_Running: No
Slave_SQL_Running: No
Replicate_Do_DB:
Replicate_Ignore_DB:
Replicate_Do_Table:
Replicate_Ignore_Table:
Replicate_Wild_Do_Table:
Replicate_Wild_Ignore_Table:
Last_Errno: 0
Last_Error:
Skip_Counter: 0
Exec_Master_Log_Pos: 862
Relay_Log_Space: 120
Until_Condition: None
Until_Log_File:
Until_Log_Pos: 0
Master_SSL_Allowed: No
Master_SSL_CA_File:
Master_SSL_CA_Path:
Master_SSL_Cert:
Master_SSL_Cipher:
Master_SSL_Key:
Seconds_Behind_Master: NULL
Master_SSL_Verify_Server_Cert: No
Last_IO_Errno: 1236
Last_IO_Error: Got fatal error 1236 from master when reading data from binary log: 'Could not find first log file name in binary log index file'
Last_SQL_Errno: 0
Last_SQL_Error:
Replicate_Ignore_Server_Ids:
Master_Server_Id: 2433306
Master_UUID: b4cf465a-8447-11e5-ac19-080027796e8a
Master_Info_File: /data/mysqldata/3307/data/master.info
SQL_Delay: 0
SQL_Remaining_Delay: NULL
Slave_SQL_Running_State:
Master_Retry_Count: 86400
Master_Bind:
Last_IO_Error_Timestamp: 160408 12:42:59
Last_SQL_Error_Timestamp:
Master_SSL_Crl:
Master_SSL_Crlpath:
Retrieved_Gtid_Set:
Executed_Gtid_Set:
Auto_Position: 0
1 row in set (0.00 sec)
(system@localhost)[(none)]>reset slave;
Query OK, 0 rows affected (0.14 sec)
(system@localhost)[(none)]>show slave status\G
*************************** 1. row ***************************
Slave_IO_State:
Master_Host: 192.168.56.2
Master_User: repl
Master_Port: 3306
Connect_Retry: 60
Master_Log_File:
Read_Master_Log_Pos: 4
Relay_Log_File: mysql-relay-bin.000001
Relay_Log_Pos: 4
Relay_Master_Log_File:
Slave_IO_Running: No
Slave_SQL_Running: No
Replicate_Do_DB:
Replicate_Ignore_DB:
Replicate_Do_Table:
Replicate_Ignore_Table:
Replicate_Wild_Do_Table:
Replicate_Wild_Ignore_Table:
Last_Errno: 0
Last_Error:
Skip_Counter: 0
Exec_Master_Log_Pos: 0
Relay_Log_Space: 143
Until_Condition: None
Until_Log_File:
Until_Log_Pos: 0
Master_SSL_Allowed: No
Master_SSL_CA_File:
Master_SSL_CA_Path:
Master_SSL_Cert:
Master_SSL_Cipher:
Master_SSL_Key:
Seconds_Behind_Master: NULL
Master_SSL_Verify_Server_Cert: No
Last_IO_Errno: 0
Last_IO_Error:
Last_SQL_Errno: 0
Last_SQL_Error:
Replicate_Ignore_Server_Ids:
Master_Server_Id: 2433306
Master_UUID: b4cf465a-8447-11e5-ac19-080027796e8a
Master_Info_File: /data/mysqldata/3307/data/master.info
SQL_Delay: 0
SQL_Remaining_Delay: NULL
Slave_SQL_Running_State:
Master_Retry_Count: 86400
Master_Bind:
Last_IO_Error_Timestamp:
Last_SQL_Error_Timestamp:
Master_SSL_Crl:
Master_SSL_Crlpath:
Retrieved_Gtid_Set:
Executed_Gtid_Set:
Auto_Position: 0
1 row in set (0.00 sec)
(system@localhost)[(none)]>start slave;
Query OK, 0 rows affected (0.14 sec)
(system@localhost)[(none)]>show slave status\G
*************************** 1. row ***************************
Slave_IO_State: Waiting for master to send event
Master_Host: 192.168.56.2
Master_User: repl
Master_Port: 3306
Connect_Retry: 60
Master_Log_File: mysql.000022
Read_Master_Log_Pos: 120
Relay_Log_File: mysql-relay-bin.000003
Relay_Log_Pos: 279
Relay_Master_Log_File: mysql.000022
Slave_IO_Running: Yes
Slave_SQL_Running: Yes
Replicate_Do_DB:
Replicate_Ignore_DB:
Replicate_Do_Table:
Replicate_Ignore_Table:
Replicate_Wild_Do_Table:
Replicate_Wild_Ignore_Table:
Last_Errno: 0
Last_Error:
Skip_Counter: 0
Exec_Master_Log_Pos: 120
Relay_Log_Space: 452
Until_Condition: None
Until_Log_File:
Until_Log_Pos: 0
Master_SSL_Allowed: No
Master_SSL_CA_File:
Master_SSL_CA_Path:
Master_SSL_Cert:
Master_SSL_Cipher:
Master_SSL_Key:
Seconds_Behind_Master: 0
Master_SSL_Verify_Server_Cert: No
Last_IO_Errno: 0
Last_IO_Error:
Last_SQL_Errno: 0
Last_SQL_Error:
Replicate_Ignore_Server_Ids:
Master_Server_Id: 2433306
Master_UUID: b4cf465a-8447-11e5-ac19-080027796e8a
Master_Info_File: /data/mysqldata/3307/data/master.info
SQL_Delay: 0
SQL_Remaining_Delay: NULL
Slave_SQL_Running_State: Slave has read all relay log; waiting for the slave I/O thread to update it
Master_Retry_Count: 86400
Master_Bind:
Last_IO_Error_Timestamp:
Last_SQL_Error_Timestamp:
Master_SSL_Crl:
Master_SSL_Crlpath:
Retrieved_Gtid_Set:
Executed_Gtid_Set:
Auto_Position: 0
1 row in set (0.00 sec)
(system@localhost)[(none)]>
复制环境恢复。
另外一种解决方案如下:
1、slave机器停止slave
mysql> slave stop;
2、master机器flush log并记录新的binlog信息:
记录master的bin的位置,例如:mysql> show mster status;
+-------------------+----------+--------------+-------------------------------------------+
| File | Position | Binlog_Do_DB | Binlog_Ignore_DB |
+-------------------+----------+--------------+-------------------------------------------+
| mysqld-bin.000010 | 106 | | information_schema,mysql |
+-------------------+----------+--------------+-------------------------------------------+
日志为mysqld-bin.000010
刷新日志:
mysql> flush logs;
mysql> show master status;
因为刷新日志file的位置会+1,即File变成为:mysqld-bin.000011
3、马上到slave执行
mysql> CHANGE MASTER TO MASTER_LOG_FILE='mysqld-bin.000011',MASTER_LOG_POS=106;
mysql> slave start;
mysql> show slave status\G;
参考文档:
http://blog.csdn.net/billfanggs/article/details/8905991
http://www.cnblogs.com/xiaoit/p/4489643.html
http://blog.csdn.net/mchdba/article/details/8990173