ProxySQL实现读写分离

ProxySQL实现读写分离示例

环境说明：

主机名称	IP地址	角色	应用	系统平台
proxysql	192.168.110.11	读写分离解析主机	proxysql	RHEL 8
master	192.168.110.12	master	mysql	RHEL 8
slave	192.168.110.13	slave	mysql	RHEL 8

准备工作

关闭防火墙和SELINUX

//三台主机都要执行
systemctl disable --now firewalld
setenforce 0
sed -ri 's/^(SELINUX=).*/1disabled/g' /etc/selinux/config

安装mysql并配置主从关系

详情配置请见：Mysql主从配置

开始部署

安装ProxySQL

//ProxySQL
#配置proxysql的yum源
[root@proxysql ~]# vim /etc/yum.repos.d/proxysql.repo
[proxysql_repo]
name=ProxySQL
baseurl=http://repo.proxysql.com/ProxySQL/proxysql-2.1.x/centos/8/
gpgcheck=0
enabled=1

#安装proxysql
[root@proxysql ~]# yum -y install proxysql

#设置proxysql开机自启，并现在自动
[root@proxysql ~]# systemctl enable --now proxysql

配置ProxySQL

mysql主库添加proxysql可以增删改查的账号

//master
[root@master ~]# mysql
Welcome to the MariaDB monitor.  Commands end with ; or g.
Your MariaDB connection id is 11
Server version: 10.3.28-MariaDB-log MariaDB Server

Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.

Type 'help;' or 'h' for help. Type 'c' to clear the current input statement.

MariaDB [(none)]> grant all on *.* to 'proxysql'@'192.168.110.13' identified by 'pwproxysql';
Query OK, 0 rows affected (0.001 sec)

MariaDB [(none)]> flush privileges;
Query OK, 0 rows affected (0.001 sec)

MariaDB [(none)]> quit
Bye

登录proxysql管理端

//proxysql
#安装Mariadb
[root@proxysql ~]# yum -y install mariadb*
[root@proxysql ~]# systemctl enable --now mariadb

#登录
[root@proxysql ~]# export MYSQL_PS1="(u@h:p) [d]> "
[root@proxysql ~]# mysql -uadmin -padmin -h127.0.0.1 -P6032
Welcome to the MariaDB monitor.  Commands end with ; or g.
Your MySQL connection id is 2
Server version: 5.5.30 (ProxySQL Admin Module)

Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.

Type 'help;' or 'h' for help. Type 'c' to clear the current input statement.

(admin@127.0.0.1:6032) [(none)]> show databases;
+-----+---------------+-------------------------------------+
| seq | name          | file                                |
+-----+---------------+-------------------------------------+
| 0   | main          |                                     |
| 2   | disk          | /var/lib/proxysql/proxysql.db       |
| 3   | stats         |                                     |
| 4   | monitor       |                                     |
| 5   | stats_history | /var/lib/proxysql/proxysql_stats.db |
+-----+---------------+-------------------------------------+
5 rows in set (0.000 sec)

数据库说明：

• main 内存配置数据库，表里存放后端db实例、用户验证、路由规则等信息。表名以 runtime开头的表示proxysql当前运行的配置内容，不能通过dml语句修改，只能修改对应的不以 runtime 开头的（在内存）里的表，然后 LOAD 使其生效， SAVE 使其存到硬盘以供下次重启加载
• disk 是持久化到硬盘的配置，sqlite数据文件
• stats 是proxysql运行抓取的统计信息，包括到后端各命令的执行次数、流量、processlist、查询种类汇总/执行时间等等
• monitor 库存储 monitor 模块收集的信息，主要是对后端db的健康/延迟检查
• stats_history 统计信息历史库

Proxysql管理端添加后端连接mysql主从数据库的配置

(admin@127.0.0.1:6032) [(none)]> show tables from main;
+--------------------------------------------+
| tables                                     |
+--------------------------------------------+
| global_variables                           |
| mysql_aws_aurora_hostgroups                |
| mysql_collations                           |
| mysql_galera_hostgroups                    |
| mysql_group_replication_hostgroups         |
| mysql_query_rules                          |
| mysql_query_rules_fast_routing             |
| mysql_replication_hostgroups               |
| mysql_servers                              |
| mysql_users                                |
| proxysql_servers                           |
| runtime_checksums_values                   |
| runtime_global_variables                   |
| runtime_mysql_aws_aurora_hostgroups        |
| runtime_mysql_galera_hostgroups            |
| runtime_mysql_group_replication_hostgroups |
| runtime_mysql_query_rules                  |
| runtime_mysql_query_rules_fast_routing     |
| runtime_mysql_replication_hostgroups       |
| runtime_mysql_servers                      |
| runtime_mysql_users                        |
| runtime_proxysql_servers                   |
| runtime_scheduler                          |
| scheduler                                  |
+--------------------------------------------+
24 rows in set (0.000 sec)

runtime_ 开头的是运行时的配置，这些是不能修改的。要修改 ProxySQL 的配置，需要修改了非 runtime_ 表，修改后必须执行 LOAD ... TO RUNTIME 才能加载到 RUNTIME 生效，执行 save ... to disk 才能将配置持久化保存到磁盘

下面语句中没有先切换到 main 库也执行成功了，因为 ProxySQL 内部使用的 SQLite3 数据库引擎，和 MySQL 的解析方式是不一样的。即使执行了 USE main 语句也是无任何效果的，但不会报错

使用 insert 语句添加 mysql 主机到 mysql_servers 表中，其中：hostgroup_id 1 表示写组，2表示读组

(admin@127.0.0.1:6032) [(none)]> insert into mysql_servers(hostgroup_id,hostname,port,weight,comment) values(1,'192.168.110.12',3306,1,'Write Group');
Query OK, 1 row affected (0.000 sec)

(admin@127.0.0.1:6032) [(none)]> insert into mysql_servers(hostgroup_id,hostname,port,weight,comment) values(2,'192.168.110.13',3306,1,'Read Group');
Query OK, 1 row affected (0.000 sec)

(admin@127.0.0.1:6032) [(none)]> select * from mysql_servers;
+--------------+---------------+------+-----------+--------+--------+-------------+-----------------+---------------------+---------+----------------+-------------+
| hostgroup_id | hostname      | port | gtid_port | status | weight | compression | max_connections | max_replication_lag | use_ssl | max_latency_ms | comment     |
+--------------+---------------+------+-----------+--------+--------+-------------+-----------------+---------------------+---------+----------------+-------------+
| 1            | 192.168.110.12 | 3306 | 0         | ONLINE | 1      | 0           | 1000            | 0                   | 0       | 0              | Write Group |
| 2            | 192.168.110.13 | 3306 | 0         | ONLINE | 1      | 0           | 1000            | 0                   | 0       | 0              | Read Group  |
+--------------+---------------+------+-----------+--------+--------+-------------+-----------------+---------------------+---------+----------------+-------------+
2 rows in set (0.000 sec)

修改后，需要加载到RUNTIME，并保存到disk

(admin@127.0.0.1:6032) [(none)]> load mysql servers to runtime;
Query OK, 0 rows affected (0.003 sec)

(admin@127.0.0.1:6032) [(none)]> save mysql servers to disk;
Query OK, 0 rows affected (0.004 sec)

在 proxysql 主机的 mysql_users 表中添加刚才在 master 上创建的账号 proxysql，proxysql 客户端需要使用这个账号来访问数据库；
default_hostgroup 默认组设置为写组，也就是1；
当读写分离的路由规则不符合时，会访问默认组的数据库。

(admin@127.0.0.1:6032) [(none)]> insert into mysql_users(username,password,default_hostgroup,transaction_persistent)values('proxysql','pwproxysql',1,1);
Query OK, 1 row affected (0.000 sec)

(admin@127.0.0.1:6032) [(none)]> select * from mysql_users G
*************************** 1. row ***************************
              username: proxysql        # 后端mysql实例的用户名
              password: pwproxysql      # 后端mysql实例的密码
                active: 1               # active=1表示用户生效，0表示不生效
               use_ssl: 0
     default_hostgroup: 1               # 用户默认登录到哪个hostgroup_id下的实例
        default_schema: NULL            # 用户默认登录后端mysql实例时连接的数据库，这个地方为NULL的话，则由全局变量mysql-default_schema决定，默认是information_schema
         schema_locked: 0
transaction_persistent: 1               # 如果设置为1，连接上ProxySQL的会话后，如果在一个hostgroup上开启了事务，那么后续的sql都继续维持在这个hostgroup上，不论是否会匹配上其它路由规则，直到事务结束。虽然默认是0
          fast_forward: 0               # 忽略查询重写/缓存层，直接把这个用户的请求透传到后端DB。相当于只用它的连接池功能，一般不用，路由规则 .* 就行了
               backend: 1
              frontend: 1
       max_connections: 10000           # 该用户允许的最大连接数
               comment: 
1 row in set (0.000 sec)

(admin@127.0.0.1:6032) [(none)]> load mysql users to runtime;
Query OK, 0 rows affected (0.000 sec)

(admin@127.0.0.1:6032) [(none)]> save mysql users to disk;
Query OK, 0 rows affected (0.002 sec)

添加健康检测的帐号

在mysql的 master 端添加属于proxysql的只读账号

//master
[root@master ~]# mysql
Welcome to the MariaDB monitor.  Commands end with ; or g.
Your MariaDB connection id is 42
Server version: 10.3.28-MariaDB-log MariaDB Server

Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.

Type 'help;' or 'h' for help. Type 'c' to clear the current input statement.

MariaDB [(none)]> GRANT SELECT ON *.* TO 'monitor'@'192.168.110.%' IDENTIFIED BY 'monitor';
Query OK, 0 rows affected (0.000 sec)

MariaDB [(none)]> flush privileges;
Query OK, 0 rows affected (0.001 sec)

在proxysql主机端修改变量设置健康检测的账号

//proxysql
(admin@127.0.0.1:6032) [(none)]> set mysql-monitor_username='monitor';
Query OK, 1 row affected (0.001 sec)

(admin@127.0.0.1:6032) [(none)]> set mysql-monitor_password='monitor';
Query OK, 1 row affected (0.000 sec)

(admin@127.0.0.1:6032) [(none)]> load mysql variables to runtime;
Query OK, 0 rows affected (0.001 sec)

(admin@127.0.0.1:6032) [(none)]> save mysql variables to disk;
Query OK, 140 rows affected (0.002 sec)

添加读写分离的路由规则

需求：

• 将 select 查询语句全部路由至 hostgroup_id=2 的组(也就是读组)
• 但是 select * from tb for update 这样的语句是会修改数据的，所以需要单独定义，将它路由至 hostgroup_id=1 的组(也就是写组)
• 其他没有被规则匹配到的组将会被路由至用户默认的组(mysql_users 表中的 default_hostgroup)

(admin@127.0.0.1:6032) [(none)]> insert into mysql_query_rules(rule_id,active,match_digest,destination_hostgroup,apply)values(1,1,'^SELECT.*FOR UPDATE$',1,1);
Query OK, 1 row affected (0.000 sec)

(admin@127.0.0.1:6032) [(none)]> insert into mysql_query_rules(rule_id,active,match_digest,destination_hostgroup,apply)values(2,1,'^SELECT',2,1);
Query OK, 1 row affected (0.000 sec)

(admin@127.0.0.1:6032) [(none)]> select rule_id,active,match_digest,destination_hostgroup,apply from mysql_query_rules;
+---------+--------+----------------------+-----------------------+-------+
| rule_id | active | match_digest         | destination_hostgroup | apply |
+---------+--------+----------------------+-----------------------+-------+
| 1       | 1      | ^SELECT.*FOR UPDATE$ | 1                     | 1     |
| 2       | 1      | ^SELECT              | 2                     | 1     |
+---------+--------+----------------------+-----------------------+-------+
2 rows in set (0.000 sec)

(admin@127.0.0.1:6032) [(none)]> load mysql query rules to runtime;
Query OK, 0 rows affected (0.001 sec)

(admin@127.0.0.1:6032) [(none)]> load admin variables to runtime;
Query OK, 0 rows affected (0.001 sec)

(admin@127.0.0.1:6032) [(none)]> save mysql query rules to disk;
Query OK, 0 rows affected (0.002 sec)

(admin@127.0.0.1:6032) [(none)]> save admin variables to disk;
Query OK, 33 rows affected (0.002 sec)

验证读写分离

登录 proxysql 客户端

登录用户是刚才我们在 mysql_user 表中创建的用户，端口为6033

[root@proxysql ~]# mysql -uproxysql -ppwproxysql -h127.0.0.1 -P6033
Welcome to the MariaDB monitor.  Commands end with ; or g.
Your MySQL connection id is 5
Server version: 5.5.30 (ProxySQL)

Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.

Type 'help;' or 'h' for help. Type 'c' to clear the current input statement.

(proxysql@127.0.0.1:6033) [(none)]> show databases;
+--------------------+
| Database           |
+--------------------+
| information_schema |
| mysql              |
| performance_schema |
+--------------------+
3 rows in set (0.003 sec)

尝试修改数据库和查询

创建2个数据库并查询一下表

(proxysql@127.0.0.1:6033) [(none)]> create database ldaz1;
Query OK, 1 row affected (0.001 sec)

(proxysql@127.0.0.1:6033) [(none)]> create database ldaz2;
Query OK, 1 row affected (0.001 sec)

(proxysql@127.0.0.1:6033) [(none)]> select user,host from mysql.user;
+----------+-----------------------+
| user     | host                  |
+----------+-----------------------+
| root     | 127.0.0.1             |
| monitor  | 192.168.110.%         |
| proxysql | 192.168.110.12         |
| root     | ::1                   |
| root     | localhost             |
| root     | localhost.localdomain |
+----------+-----------------------+
6 rows in set (0.002 sec)

(proxysql@127.0.0.1:6033) [(none)]> quit
Bye

验证读写分离是否成功

proxysql有个类似审计的功能，可以查看各类SQL的执行情况，其需要在proxysql管理端执行

[root@proxysql ~]# mysql -uadmin -padmin -h127.0.0.1 -P6032
Welcome to the MariaDB monitor.  Commands end with ; or g.
Your MySQL connection id is 6
Server version: 5.5.30 (ProxySQL Admin Module)

Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.

Type 'help;' or 'h' for help. Type 'c' to clear the current input statement.

(admin@127.0.0.1:6032) [(none)]> select * from stats_mysql_query_digest;
+-----------+--------------------+----------+----------------+--------------------+----------------------------------+------------+------------+------------+----------+----------+----------+-------------------+---------------+
| hostgroup | schemaname         | username | client_address | digest             | digest_text                      | count_star | first_seen | last_seen  | sum_time | min_time | max_time | sum_rows_affected | sum_rows_sent |
+-----------+--------------------+----------+----------------+--------------------+----------------------------------+------------+------------+------------+----------+----------+----------+-------------------+---------------+
| 1         | information_schema | proxysql |                | 0x8DE5F3CF3D5041DF | create database ldaz2           | 1          | 1622733165 | 1622733165 | 727      | 727      | 727      | 1                 | 0             |
| 1         | information_schema | proxysql |                | 0xCF3CF194C63452E1 | create database ldaz1            | 1          | 1622733161 | 1622733161 | 655      | 655      | 655      | 1                 | 0             |
| 2         | information_schema | proxysql |                | 0x0F02B330C823D739 | select user,host from mysql.user | 1          | 1622733170 | 1622733170 | 1839     | 1839     | 1839     | 0                 | 6             |
+-----------+--------------------+----------+----------------+--------------------+----------------------------------+------------+------------+------------+----------+----------+----------+-------------------+---------------+
3 rows in set (0.002 sec)

从上面的 hostgroup 和 digest_text 值来看，所有的写操作都被路由至1组，读操作都被路由至2组，其中1组为写组，2组为读组！