ProxySQL Cluster的搭建
环境:
proxysql-1.4.10-1-centos7.x86_64
db210 192.168.99.210 老节点,已经做成mysql配置和读写分离设置
db211 192.168.99.211 新节点
db212 192.168.99.212 新节点
1.设置集群管理账号
在所有节点上做以下操作
admin@127.0.0.1 [man]>update global_variables set variable_value='admin:admin;cluster_21x:zstxxx' where variable_name='admin-admin_credentials';
Query OK, 1 row affected (0.00 sec)
admin@127.0.0.1 [man]>update global_variables set variable_value='cluster_21x' where variable_name='admin-cluster_username';
Query OK, 1 row affected (0.00 sec)
admin@127.0.0.1 [man]>update global_variables set variable_value='zstxxx' where variable_name='admin-cluster_password';
2.设置集群成员:
admin@127.0.0.1 [man]>insert into proxysql_servers(hostname,port,weight,comment) values('db210',6032,1,'primary'),('db211',6032,1,'secondary'),('db212',6032,1,'secondary');
3.加载配置
- 先加载已有服务器配置的第一个ProxySQL节点,再加载其他新节点.
admin@127.0.0.1 [man]>load admin variables to runtime;
admin@127.0.0.1 [man]>load proxysql servers to runtime;
admin@127.0.0.1 [man]>save admin variables to disk;
admin@127.0.0.1 [man]>save proxysql servers to disk;
4.查看群集状态:
注:节点db212还没有配置,所以状态为空
admin@127.0.0.1 [(none)]>select hostname,port,comment,Uptime_s,last_check_ms from stats_proxysql_servers_metrics;
+----------+------+-----------+----------+---------------+
| hostname | port | comment | Uptime_s | last_check_ms |
+----------+------+-----------+----------+---------------+
| db212 | 6032 | secondary | 1064 | 468 |
| db211 | 6032 | secondary | 100320 | 279 |
| db210 | 6032 | primary | 7300 | 283 |
+----------+------+-----------+----------+---------------+
3 rows in set (0.00 sec)
admin@127.0.0.1 [(none)]>select hostname,name,checksum,updated_at from stats_proxysql_servers_checksums;
+----------+-------------------+--------------------+------------+
| hostname | name | checksum | updated_at |
+----------+-------------------+--------------------+------------+
| db212 | admin_variables | | 1535100518 |
| db212 | mysql_query_rules | 0x0000000000000000 | 1535100518 |
| db212 | mysql_servers | 0xB58B50EC4FEF3D20 | 1535100518 |
| db212 | mysql_users | 0x0000000000000000 | 1535100518 |
| db212 | mysql_variables | | 1535100518 |
| db212 | proxysql_servers | 0xC405ABD157E8B011 | 1535100518 |
| db211 | admin_variables | | 1535100518 |
| db211 | mysql_query_rules | 0x0000000000000000 | 1535100518 |
| db211 | mysql_servers | 0xB58B50EC4FEF3D20 | 1535100518 |
| db211 | mysql_users | 0x0000000000000000 | 1535100518 |
| db211 | mysql_variables | | 1535100518 |
| db211 | proxysql_servers | 0xC405ABD157E8B011 | 1535100518 |
| db210 | admin_variables | | 1535100518 |
| db210 | mysql_query_rules | 0xC2127DA38220E59D | 1535100518 |
| db210 | mysql_servers | 0x2E65DC53740DC1C8 | 1535100518 |
| db210 | mysql_users | 0x9881C192AE6F7368 | 1535100518 |
| db210 | mysql_variables | | 1535100518 |
| db210 | proxysql_servers | 0xC405ABD157E8B011 | 1535100518 |
+----------+-------------------+--------------------+------------+
18 rows in set (0.00 sec)
5.数据同步的触发
集群建立后,发现有部分数据没有同步,如下表中可以看出db211和db212的mysql_query_rules,mysql_users的checksum都不对
admin@127.0.0.1 [(none)]>select hostname,name,checksum,updated_at from stats_proxysql_servers_checksums;
+----------+-------------------+--------------------+------------+
| hostname | name | checksum | updated_at |
+----------+-------------------+--------------------+------------+
| db212 | admin_variables | | 1535100518 |
| db212 | mysql_query_rules | 0x0000000000000000 | 1535100518 |
| db212 | mysql_servers | 0xB58B50EC4FEF3D20 | 1535100518 |
| db212 | mysql_users | 0x0000000000000000 | 1535100518 |
| db212 | mysql_variables | | 1535100518 |
| db212 | proxysql_servers | 0xC405ABD157E8B011 | 1535100518 |
| db211 | admin_variables | | 1535100518 |
| db211 | mysql_query_rules | 0x0000000000000000 | 1535100518 |
| db211 | mysql_servers | 0xB58B50EC4FEF3D20 | 1535100518 |
| db211 | mysql_users | 0x0000000000000000 | 1535100518 |
| db211 | mysql_variables | | 1535100518 |
| db211 | proxysql_servers | 0xC405ABD157E8B011 | 1535100518 |
| db210 | admin_variables | | 1535100518 |
| db210 | mysql_query_rules | 0xC2127DA38220E59D | 1535100518 |
| db210 | mysql_servers | 0x2E65DC53740DC1C8 | 1535100518 |
| db210 | mysql_users | 0x9881C192AE6F7368 | 1535100518 |
| db210 | mysql_variables | | 1535100518 |
| db210 | proxysql_servers | 0xC405ABD157E8B011 | 1535100518 |
+----------+-------------------+--------------------+------------+
18 rows in set (0.00 sec)
- 查日志发现以下信息:
detected a peer db211:6032 with mysql_users version 1, epoch 1535000044, diff_check 7860. Own version: 1, epoch: 1535093184. diff_check is increasing, but version 1 doesn't allow sync. This message will be repeated every 30 checks until LOAD MYSQL USERS TO RUNTIME is executed on candidate master.
- 解决办法:在主节点上load to RUNTIME
6.遗留问题:
- 发现admin_variables和mysql_variables 的 checksum 始终为空,修改值也无法同步.
- 集群同步监测表中没有mysql_group_replication_hostgroups,还需要自行同步.
- db212节点的mysql_servers表的checksum 不同,但肉眼只能看出记录顺序不同,导致一致产生大量日志,尝试重新同步很多次,都出现相同的结果.
db210主节点:
admin@127.0.0.1 [man]>select hostname,name,checksum,changed_at,diff_check from stats_proxysql_servers_checksums;
+----------+-------------------+--------------------+------------+------------+
| hostname | name | checksum | changed_at | diff_check |
+----------+-------------------+--------------------+------------+------------+
| db212 | admin_variables | | 0 | 0 |
| db212 | mysql_query_rules | 0xC2127DA38220E59D | 1535122622 | 0 |
| db212 | mysql_servers | 0xB58B50EC4FEF3D20 | 1535122622 | 110 |
| db212 | mysql_users | 0x6C619C922DBF30FF | 1535122622 | 0 |
| db212 | mysql_variables | | 0 | 0 |
| db212 | proxysql_servers | 0xC405ABD157E8B011 | 1535122623 | 0 |
| db211 | admin_variables | | 0 | 0 |
| db211 | mysql_query_rules | 0xC2127DA38220E59D | 1535111313 | 0 |
| db211 | mysql_servers | 0x2E65DC53740DC1C8 | 1535102178 | 0 |
| db211 | mysql_users | 0x6C619C922DBF30FF | 1535105932 | 0 |
| db211 | mysql_variables | | 0 | 0 |
| db211 | proxysql_servers | 0xC405ABD157E8B011 | 1535122626 | 0 |
| db210 | admin_variables | | 0 | 0 |
| db210 | mysql_query_rules | 0xC2127DA38220E59D | 1535096359 | 0 |
| db210 | mysql_servers | 0x2E65DC53740DC1C8 | 1535100019 | 0 |
| db210 | mysql_users | 0x6C619C922DBF30FF | 1535105930 | 0 |
| db210 | mysql_variables | | 0 | 0 |
| db210 | proxysql_servers | 0xC405ABD157E8B011 | 1535122622 | 0 |
+----------+-------------------+--------------------+------------+------------+
18 rows in set (0.00 sec)
admin@127.0.0.1 [man]>select * from mysql_servers;
+--------------+----------+------+--------+--------+-------------+-----------------+---------------------+---------+----------------+---------+
| hostgroup_id | hostname | port | status | weight | compression | max_connections | max_replication_lag | use_ssl | max_latency_ms | comment |
+--------------+----------+------+--------+--------+-------------+-----------------+---------------------+---------+----------------+---------+
| 10 | db212 | 3507 | ONLINE | 1 | 0 | 1000 | 0 | 0 | 0 | |
| 11 | db210 | 3507 | ONLINE | 1 | 0 | 1000 | 0 | 0 | 0 | |
| 11 | db212 | 3507 | ONLINE | 1 | 0 | 1000 | 0 | 0 | 0 | |
| 11 | db211 | 3507 | ONLINE | 1 | 0 | 1000 | 0 | 0 | 0 | |
| 13 | db210 | 3507 | ONLINE | 1 | 0 | 1000 | 0 | 0 | 0 | |
| 13 | db211 | 3507 | ONLINE | 1 | 0 | 1000 | 0 | 0 | 0 | |
+--------------+----------+------+--------+--------+-------------+-----------------+---------------------+---------+----------------+---------+
6 rows in set (0.00 sec)
- db212节点
admin@127.0.0.1 [(none)]>select * from disk.mysql_servers;
+--------------+----------+------+--------+--------+-------------+-----------------+---------------------+---------+----------------+---------+
| hostgroup_id | hostname | port | status | weight | compression | max_connections | max_replication_lag | use_ssl | max_latency_ms | comment |
+--------------+----------+------+--------+--------+-------------+-----------------+---------------------+---------+----------------+---------+
| 10 | db212 | 3507 | ONLINE | 1 | 0 | 1000 | 0 | 0 | 0 | |
| 13 | db211 | 3507 | ONLINE | 1 | 0 | 1000 | 0 | 0 | 0 | |
| 13 | db210 | 3507 | ONLINE | 1 | 0 | 1000 | 0 | 0 | 0 | |
| 11 | db211 | 3507 | ONLINE | 1 | 0 | 1000 | 0 | 0 | 0 | |
| 11 | db212 | 3507 | ONLINE | 1 | 0 | 1000 | 0 | 0 | 0 | |
| 11 | db210 | 3507 | ONLINE | 1 | 0 | 1000 | 0 | 0 | 0 | |
+--------------+----------+------+--------+--------+-------------+-----------------+---------------------+---------+----------------+---------+
6 rows in set (0.00 sec)
7.全配置信息
admin@127.0.0.1 [man]>select * from runtime_global_variables;
+-----------------------------------------------------+--------------------------------+
| variable_name | variable_value |
+-----------------------------------------------------+--------------------------------+
| admin-admin_credentials | admin:adminxxx;cluster_210:zstxxx |
| admin-stats_credentials | stats:stats |
| admin-stats_mysql_connections | 60 |
| admin-stats_mysql_connection_pool | 60 |
| admin-stats_mysql_query_cache | 60 |
| admin-stats_system_cpu | 60 |
| admin-stats_system_memory | 60 |
| admin-mysql_ifaces | 0.0.0.0:6032 |
| admin-telnet_admin_ifaces | (null) |
| admin-telnet_stats_ifaces | (null) |
| admin-refresh_interval | 2000 |
| admin-read_only | false |
| admin-hash_passwords | true |
| admin-version | v1.4.10-1-g5eb0f3e |
| admin-cluster_username | cluster_210 |
| admin-cluster_password | zstxxx |
| admin-cluster_check_interval_ms | 1000 |
| admin-cluster_check_status_frequency | 10 |
| admin-cluster_mysql_query_rules_diffs_before_sync | 3 |
| admin-cluster_mysql_servers_diffs_before_sync | 3 |
| admin-cluster_mysql_users_diffs_before_sync | 3 |
| admin-cluster_proxysql_servers_diffs_before_sync | 3 |
| admin-cluster_mysql_query_rules_save_to_disk | true |
| admin-cluster_mysql_servers_save_to_disk | true |
| admin-cluster_mysql_users_save_to_disk | true |
| admin-cluster_proxysql_servers_save_to_disk | true |
| admin-checksum_mysql_query_rules | true |
| admin-checksum_mysql_servers | true |
| admin-checksum_mysql_users | true |
| admin-web_enabled | false |
| admin-web_port | 6080 |
| mysql-shun_on_failures | 5 |
| mysql-shun_recovery_time_sec | 10 |
| mysql-query_retries_on_failure | 1 |
| mysql-client_multi_statements | true |
| mysql-connect_retries_on_failure | 10 |
| mysql-connect_retries_delay | 1 |
| mysql-connection_delay_multiplex_ms | 0 |
| mysql-connection_max_age_ms | 0 |
| mysql-connect_timeout_server | 3000 |
| mysql-connect_timeout_server_max | 10000 |
| mysql-eventslog_filename | |
| mysql-eventslog_filesize | 104857600 |
| mysql-default_charset | utf8 |
| mysql-free_connections_pct | 10 |
| mysql-session_idle_ms | 1000 |
| mysql-have_compress | true |
| mysql-client_found_rows | true |
| mysql-interfaces | 0.0.0.0:6033 |
| mysql-monitor_enabled | true |
| mysql-monitor_history | 600000 |
| mysql-monitor_connect_interval | 60000 |
| mysql-monitor_connect_timeout | 600 |
| mysql-monitor_ping_interval | 10000 |
| mysql-monitor_ping_max_failures | 3 |
| mysql-monitor_ping_timeout | 1000 |
| mysql-monitor_read_only_interval | 1500 |
| mysql-monitor_read_only_timeout | 500 |
| mysql-monitor_read_only_max_timeout_count | 3 |
| mysql-monitor_replication_lag_interval | 10000 |
| mysql-monitor_replication_lag_timeout | 1000 |
| mysql-monitor_groupreplication_healthcheck_interval | 5000 |
| mysql-monitor_groupreplication_healthcheck_timeout | 800 |
| mysql-monitor_username | proxysql |
| mysql-monitor_password | zstxxx |
| mysql-monitor_replication_lag_use_percona_heartbeat | |
| mysql-monitor_query_interval | 60000 |
| mysql-monitor_query_timeout | 100 |
| mysql-monitor_slave_lag_when_null | 60 |
| mysql-monitor_wait_timeout | true |
| mysql-monitor_writer_is_also_reader | true |
| mysql-max_allowed_packet | 4194304 |
| mysql-throttle_connections_per_sec_to_hostgroup | 1000000 |
| mysql-max_transaction_time | 14400000 |
| mysql-multiplexing | true |
| mysql-forward_autocommit | false |
| mysql-enforce_autocommit_on_reads | false |
| mysql-autocommit_false_not_reusable | false |
| mysql-autocommit_false_is_transaction | false |
| mysql-verbose_query_error | false |
| mysql-hostgroup_manager_verbose | 1 |
| mysql-threshold_query_length | 524288 |
| mysql-threshold_resultset_size | 4194304 |
| mysql-query_digests_max_digest_length | 2048 |
| mysql-query_digests_max_query_length | 65000 |
| mysql-wait_timeout | 28800000 |
| mysql-throttle_max_bytes_per_second_to_client | 2147483647 |
| mysql-throttle_ratio_server_to_client | 0 |
| mysql-max_connections | 2048 |
| mysql-max_stmts_per_connection | 20 |
| mysql-max_stmts_cache | 10000 |
| mysql-mirror_max_concurrency | 16 |
| mysql-mirror_max_queue_length | 32000 |
| mysql-default_max_latency_ms | 1000 |
| mysql-default_query_delay | 0 |
| mysql-default_query_timeout | 36000000 |
| mysql-query_processor_iterations | 0 |
| mysql-query_processor_regex | 1 |
| mysql-long_query_time | 1000 |
| mysql-query_cache_size_MB | 256 |
| mysql-ping_interval_server_msec | 120000 |
| mysql-ping_timeout_server | 500 |
| mysql-default_schema | information_schema |
| mysql-poll_timeout | 2000 |
| mysql-poll_timeout_on_failure | 100 |
| mysql-server_capabilities | 45578 |
| mysql-server_version | 5.5.30 |
| mysql-kill_backend_connection_when_disconnect | true |
| mysql-sessions_sort | true |
| mysql-session_idle_show_processlist | true |
| mysql-commands_stats | true |
| mysql-query_digests | true |
| mysql-query_digests_lowercase | false |
| mysql-servers_stats | true |
| mysql-default_reconnect | true |
| mysql-ssl_p2s_ca | |
| mysql-ssl_p2s_cert | |
| mysql-ssl_p2s_key | |
| mysql-ssl_p2s_cipher | |
| mysql-stacksize | 1048576 |
| mysql-threads | 4 |
| mysql-init_connect | |
| mysql-default_sql_mode | |
| mysql-default_time_zone | SYSTEM |
| mysql-connpoll_reset_queue_length | 50 |
| mysql-stats_time_backend_query | false |
| mysql-stats_time_query_processor | false |
+-----------------------------------------------------+--------------------------------+
127 rows in set (0.00 sec)
admin@127.0.0.1 [man]>
admin@127.0.0.1 [man]>select * from runtime_proxysql_servers;
+----------+------+--------+-----------+
| hostname | port | weight | comment |
+----------+------+--------+-----------+
| db210 | 6032 | 1 | primary |
| db211 | 6032 | 1 | secondary |
| db212 | 6032 | 1 | secondary |
+----------+------+--------+-----------+
3 rows in set (0.00 sec)