Icident event 分析

现象

备库中断，显示如下错误

      Connect_Retry: 60
              Master_Log_File: mysql-bin.000185
          Read_Master_Log_Pos: 308647804
               Relay_Log_File: slave-relay.000002
                Relay_Log_Pos: 1456
        Relay_Master_Log_File: mysql-bin.000119
             Slave_IO_Running: Yes
            Slave_SQL_Running: No
              Replicate_Do_DB: 
          Replicate_Ignore_DB: 
           Replicate_Do_Table: 
       Replicate_Ignore_Table: 
      Replicate_Wild_Do_Table: 
  Replicate_Wild_Ignore_Table: 
                   Last_Errno: 1590
                   Last_Error: The incident LOST_EVENTS occured on the master. Message: error writing to the binary log
                 Skip_Counter: 0
          Exec_Master_Log_Pos: 525863559
              Relay_Log_Space: 35663403551

　　

error log日志如下

2016-06-08 15:40:27 22134 [ERROR] The incident LOST_EVENTS occured on the master. Message: error writing to the binary log, Error_code: 1590

查看binlog

mysqlbinlog -vv slave-relay.000002 --start-position=1456|more

# at 1456
#160604 19:18:57 server id 1129472893  end_log_pos 525863616 CRC32 0xece8422b 
# Incident: LOST_EVENTS
RELOAD DATABASE; # Shall generate syntax error

Incident: LOST_EVENTS 是从主库拉取来

分析

主库写INCIDENT_EVENT的时机

1. 权限变更操作可能只处理了一部分并发生错误时，会写一条INCIDENT_EVENT
2. 事务中包含非事务表操作，但中途写binlog到cache出错会写一条INCIDENT_EVENT

以上两种情况都会造成binlog复制出现不一致，因此主库选择记录INCIDENT_EVENT，备库在解析到 Incident event就直接报错

Incident_log_event::do_apply_event(Relay_log_info const *rli)
{   
DBUG_ENTER("Incident_log_event::do_apply_event");

if (ignored_error_code(ER_SLAVE_INCIDENT))
{
 DBUG_PRINT("info", ("Ignoring Incident"));
 DBUG_RETURN(0);
}           

rli->report(ERROR_LEVEL, ER_SLAVE_INCIDENT,
           ER(ER_SLAVE_INCIDENT),
           description(),
           m_message.length > 0 ? m_message.str : "<none>");
DBUG_RETURN(1);
}

　　

修复

  此种情况一般建议备库重搭

  相关bug https://bugs.mysql.com/bug.php?id=68892 已修复