ProxySQL（MGR） 故障排查：
- 故障现象：runtime_mysql_servers节点状态offline_hostgroup（本案例为15）
- 日志关键信息:
[WARNING] Group Replication: setting host db211:3507 offline because: Table 'sys.gr_member_routing_candidate_status' doesn't exist
具体情况如下:
admin@127.0.0.1 [man]>select * from runtime_mysql_group_replication_hostgroups;
+------------------+-------------------------+------------------+----------------+--------+------------+----+
| writer_hostgroup | backup_writer_hostgroup | reader_hostgroup | offline_hostgroup | active | max_writers | writer_is_also_reader |
+------------------+-------------------------+------------------+----------------+--------+------------+----+
| 10 | 13 | 11 | 15 | 1 | 1 | 1 |
+------------------+-------------------------+------------------+----------------+--------+------------+----+
1 row in set (0.00 sec)
admin@127.0.0.1 [man]>select * from mysql_servers;
+--------------+----------+------+--------+--------+-------------+-----------------+
| hostgroup_id | hostname | port | status | weight | compression | max_connections |
+--------------+----------+------+--------+--------+-------------+-----------------+
| 10 | db210 | 3507 | ONLINE | 1 | 0 | 1000 |
| 11 | db211 | 3507 | ONLINE | 1 | 0 | 1000 |
| 11 | db212 | 3507 | ONLINE | 1 | 0 | 1000 |
+--------------+----------+------+--------+--------+-------------+-----------------+
3 rows in set (0.00 sec)
admin@127.0.0.1 [man]>load mysql servers to runtime;
Query OK, 0 rows affected (0.01 sec)
admin@127.0.0.1 [man]>select * from runtime_mysql_servers;
+--------------+----------+------+--------+--------+-------------+-----------------+
| hostgroup_id | hostname | port | status | weight | compression | max_connections |
+--------------+----------+------+--------+--------+-------------+-----------------+
| 15 | db210 | 3507 | ONLINE | 1 | 0 | 1000 |
| 11 | db211 | 3507 | ONLINE | 1 | 0 | 1000 |
| 15 | db212 | 3507 | ONLINE | 1 | 0 | 1000 |
+--------------+----------+------+--------+--------+-------------+-----------------+
3 rows in set (0.01 sec)
- 查看proxysql.log文件:
[root@db210_21:24:47 /var/lib/proxysql]
#vi proxysql.log
2018-08-23 21:22:41 MySQL_HostGroups_Manager.cpp:2382:update_group_replication_set_offline(): [WARNING] Group Replication: setting host db210:3507 offline because: Table 'sys.gr_member_routing_candidate_status' doesn't exist
2018-08-23 21:22:41 MySQL_HostGroups_Manager.cpp:2382:update_group_replication_set_offline(): [WARNING] Group Replication: setting host db212:3507 offline because: Table 'sys.gr_member_routing_candidate_status' doesn't exist
2018-08-23 21:22:46 MySQL_HostGroups_Manager.cpp:2382:update_group_replication_set_offline(): [WARNING] Group Replication: setting host db211:3507 offline because: Table 'sys.gr_member_routing_candidate_status' doesn't exist
2018-08-23 21:22:46 MySQL_HostGroups_Manager.cpp:602:commit(): [WARNING] Removed server at address 140216413326592, hostgroup 11, address db211 port 3507. Setting status OFFLINE HARD and immediately dropping all free connections. Used connections will be dropped when trying to use them
- 经查,依赖于mysql_gr_routing_check 下载其addition_to_sys.sql导入节点中
root@localhost [sys]>source /opt/mysql_gr_routing_check/addition_to_sys.sql
Database changed
Query OK, 0 rows affected (0.00 sec)
...
Query OK, 0 rows affected (0.00 sec)
root@localhost [sys]>
- 切换到proxySQL中查看runtime_mysql_servers 已经正常:
mysql -h 127.0.0.1 -P 6032 -uadmin -padmin (wd: /data/backup)
admin@127.0.0.1 [man]>select hostgroup_id,hostname,port,status,weight,max_connections from runtime_mysql_servers;
+--------------+----------+------+--------+--------+-----------------+
| hostgroup_id | hostname | port | status | weight | max_connections |
+--------------+----------+------+--------+--------+-----------------+
| 10 | db212 | 3507 | ONLINE | 1 | 1000 |
| 13 | db211 | 3507 | ONLINE | 1 | 1000 |
| 13 | db210 | 3507 | ONLINE | 1 | 1000 |
| 11 | db212 | 3507 | ONLINE | 1 | 1000 |
| 11 | db211 | 3507 | ONLINE | 1 | 1000 |
| 11 | db210 | 3507 | ONLINE | 1 | 1000 |
+--------------+----------+------+--------+--------+-----------------+
6 rows in set (0.00 sec)