ProxySQL

https://github.com/sysown/proxysql/wiki
https://github.com/sysown/proxysql/wiki/ProxySQL-Configuration
关于ProxySQL的文档请见：https://github.com/sysown/ProxySQL

下载安装proxysql
下载地址如下：
https://github.com/sysown/ProxySQL
或者
https://www.percona.com/downloads/proxysql/

 

安装依赖包
# yum install -y perl-DBI perl-DBD-MySQL perl-Time-HiRes perl-IO-Socket-SSL
# vim /etc/yum.repos.d/proxysql.repo
[proxysql_repo]
name= ProxySQL YUM repository
baseurl=http://repo.proxysql.com/ProxySQL/proxysql-1.4.x/centos/$releasever
gpgcheck=1
gpgkey=http://repo.proxysql.com/ProxySQL/repo_pub_key
# yum install -y proxysql

 

查看proxysql涉及到哪些文件
# rpm -ql proxysql
/etc/init.d/proxysql
/etc/proxysql.cnf
/usr/bin/proxysql
/usr/share/proxysql/tools/proxysql_galera_checker.sh
/usr/share/proxysql/tools/proxysql_galera_writer.pl

 

# systemctl status proxysql.service
● proxysql.service - LSB: High Performance Advanced Proxy for MySQL
   Loaded: loaded (/etc/rc.d/init.d/proxysql; bad; vendor preset: disabled)
   Active: inactive (dead)
     Docs: man:systemd-sysv-generator(8)

 

/etc/init.d/proxysql脚本涉及到如下目录、文件
OLDDATADIR="/var/run/proxysql"
DATADIR="/var/lib/proxysql"
OPTS="-c /etc/proxysql.cnf -D $DATADIR"
PIDFILE="$DATADIR/proxysql.pid"

# more /run/systemd/generator.late/proxysql.service
# Automatically generated by systemd-sysv-generator

[Unit]
Documentation=man:systemd-sysv-generator(8)
SourcePath=/etc/rc.d/init.d/proxysql
Description=LSB: High Performance Advanced Proxy for MySQL
Before=runlevel2.target
Before=runlevel3.target
Before=runlevel4.target
Before=runlevel5.target
Before=shutdown.target
After=network-online.target
After=vmware-tools.service
After=vmware-tools-thinprint.service
Conflicts=shutdown.target

[Service]
Type=forking
Restart=no
TimeoutSec=5min
IgnoreSIGPIPE=no
KillMode=process
GuessMainPID=no
RemainAfterExit=yes
ExecStart=/etc/rc.d/init.d/proxysql start
ExecStop=/etc/rc.d/init.d/proxysql stop
ExecReload=/etc/rc.d/init.d/proxysql reload

检查版本
# which proxysql
/usr/bin/proxysql

# proxysql --version
ProxySQL version v1.4.10-1-g5eb0f3e, codename Truls

# proxysql --help
High Performance Advanced Proxy for MySQL

USAGE: proxysql [OPTIONS]

OPTIONS:

-c, --config ARG             Configuraton file
-D, --datadir ARG            Datadir
-e, --exit-on-error          Do not restart ProxySQL if crashes
-f, --foreground             Run in foreground
-h, -help, --help, --usage   Display usage instructions.
-M, --no-monitor             Do not start Monitor Module
-n, --no-start               Starts only the admin service
-r, --reuseport              Use SO_REUSEPORT
-S, --admin-socket ARG       Administration Unix Socket
-V, --version                Print version
--idle-threads               Create auxiliary threads to handle idle connections
--initial                    Rename/empty database file
--reload                     Merge config file into database file
--sqlite3-server             Enable SQLite3 Server

ProxySQL rev. v1.4.10-1-g5eb0f3e -- Tue Aug  7 12:31:55 2018
Copyright (C) 2013-2018 ProxySQL LLC
This program is free and without warranty

配置文件/etc/proxysql.cnf
配置文件只在第一次启动的时候读取进行初始化，后面只读取db文件
先启动，然后再修改参数

启动proxysql
# systemctl start proxysql
# ps -ef | grep proxysql
root       3157      1  0 01:21 ?        00:00:00 proxysql -c /etc/proxysql.cnf -D /var/lib/proxysql
root       3158   3157  1 01:21 ?        00:00:00 proxysql -c /etc/proxysql.cnf -D /var/lib/proxysql
root       3183   2821  0 01:21 pts/0    00:00:00 grep --color=auto proxysql
# netstat -antp | grep proxysql
tcp        0      0 0.0.0.0:6032            0.0.0.0:*               LISTEN      3158/proxysql      
tcp        0      0 0.0.0.0:6033            0.0.0.0:*               LISTEN      3158/proxysql

 

6032是管理端口
6033是对外服务端口
默认管理用户及密码：admin,admin

 

连接proxysql 6032管理端口
# mysql -uadmin -padmin -h 127.0.0.1 -P6032
mysql: [Warning] Using a password on the command line interface can be insecure.
Welcome to the MySQL monitor.  Commands end with ; or g.
Your MySQL connection id is 1
Server version: 5.5.30 (ProxySQL Admin Module)

Copyright (c) 2000, 2018, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or 'h' for help. Type 'c' to clear the current input statement.

(admin@127.0.0.1) [(none)]> show databases;
+-----+---------------+-------------------------------------+
| seq   | name          | file                                |
+-----+---------------+-------------------------------------+
| 0   | main             |                                                                   |
| 2   | disk               | /var/lib/proxysql/proxysql.db              |
| 3   | stats              |                                                                  |
| 4   | monitor         |                                                                 |
| 5   | stats_history | /var/lib/proxysql/proxysql_stats.db    |
+-----+---------------+-------------------------------------+
5 rows in set (0.01 sec)

main数据库
内存配置数据库，即MEMORY，表里存放后端db实例、用户验证、路由规则等信息
表名以runtime_开头的表示ProxySQL当前运行的配置内容，不能通过DML语句修改
只能修改对应的非runtime开头的表，然后load使其生效，save使其存到硬盘以供下次重启加载
(admin@127.0.0.1) [(none)]> use main;
Database changed
(admin@127.0.0.1) [main]> show tables;
+--------------------------------------------+
| tables                                     |
+--------------------------------------------+
| global_variables                            |
| mysql_collations                            |
| mysql_group_replication_hostgroups         |
| mysql_query_rules                           |
| mysql_query_rules_fast_routing              |
| mysql_replication_hostgroups                |
| mysql_servers                               |
| mysql_users                                 |
| proxysql_servers                            |
| runtime_checksums_values                    |
| runtime_global_variables                    |
| runtime_mysql_group_replication_hostgroups |
| runtime_mysql_query_rules                   |
| runtime_mysql_query_rules_fast_routing     |
| runtime_mysql_replication_hostgroups       |
| runtime_mysql_servers                       |
| runtime_mysql_users                         |
| runtime_proxysql_servers                    |
| runtime_scheduler                           |
| scheduler                                    |
+--------------------------------------------+
20 rows in set (0.00 sec)

 

global_variables参数设置，通过set来设置

mysql_query_rules：
指定Query路由到后端不同服务器的规则列表

mysql_replication_hostgroups：
监视指定主机组中所有服务器的read_only值，并且根据read_only的值将服务器分配给写入器或读取器主机组

mysql_servers：
后端可以连接MySQL服务器的列表

mysql_users：
配置后端数据库的账号和监控的账号

 

disk数据库
持久化到硬盘的配置，sqlite数据文件

stats数据库
proxysql运行抓取的统计信息，包括到后端各命令的执行次数、流量、processlist、查询种类汇总、执行时间等等
(admin@127.0.0.1) [stats]> show tables from stats;
+--------------------------------------+
| tables                               |
+--------------------------------------+
| global_variables                     |
| stats_memory_metrics                 |
| stats_mysql_commands_counters        |
| stats_mysql_connection_pool          |
| stats_mysql_connection_pool_reset    |
| stats_mysql_global                   |
| stats_mysql_prepared_statements_info |
| stats_mysql_processlist              |
| stats_mysql_query_digest             |
| stats_mysql_query_digest_reset       |
| stats_mysql_query_rules              |
| stats_mysql_users                    |
| stats_proxysql_servers_checksums     |
| stats_proxysql_servers_metrics       |
| stats_proxysql_servers_status        |
+--------------------------------------+
15 rows in set (0.00 sec)

 

monitor数据库
存储monitor模块收集的信息，主要是对后端db的健康、延迟检查
(admin@127.0.0.1) [main]> show tables from monitor;
+------------------------------------+
| tables                             |
+------------------------------------+
| mysql_server_connect_log           |
| mysql_server_group_replication_log |
| mysql_server_ping_log              |
| mysql_server_read_only_log         |
| mysql_server_replication_lag_log   |
+------------------------------------+
5 rows in set (0.00 sec)

 

基本配置
配置有三个层次：runtime<=>memory<=>disk/config file
RUNTIME：代表proxysql当前生效的正在使用的配置，无法直接修改这里的配置，必须要从下一层load进来
MEMORY：MEMORY这一层上面连接RUNTIME层，下面连接持久层。在这层可以正常操作proxysql配置，随便修改，不会影响生产环境
修改一个配置一般都是先在MEMORY层完成，然后确认正常之后再加载到RUNTIME和持久层到磁盘上
DISK和CONFIG FILE：持久化配置信息，重启后内存中的配置信息会丢失，所以需要将配置信息保留在磁盘中。重启时，可以从磁盘快速加载回来

 

添加mysql服务器列表
就是插入数据到mysql_servers
(admin@127.0.0.1) [main]> insert into mysql_servers(hostgroup_id,hostname,port) values (1,'192.168.1.101',3306);
Query OK, 1 row affected (0.00 sec)
(admin@127.0.0.1) [main]> insert into mysql_servers(hostgroup_id,hostname,port) values (1,'192.168.1.102',3306);
Query OK, 1 row affected (0.00 sec)
(admin@127.0.0.1) [main]> insert into mysql_servers(hostgroup_id,hostname,port) values (1,'192.168.1.103',3306);
Query OK, 1 row affected (0.00 sec)
(admin@127.0.0.1) [main]> load mysql servers to runtime;
Query OK, 0 rows affected (0.00 sec)
(admin@127.0.0.1) [main]> save mysql servers to disk;
Query OK, 0 rows affected (0.03 sec)
(admin@127.0.0.1) [main]> select * from main.mysql_servers;
+--------------+---------------+------+--------+--------+-------------+-----------------+---------------------+---------+----------------+---------+
| hostgroup_id | hostname      | port | status | weight | compression | max_connections | max_replication_lag | use_ssl | max_latency_ms | comment |
+--------------+---------------+------+--------+--------+-------------+-----------------+---------------------+---------+----------------+---------+
| 1            | 192.168.1.101 | 3306 | ONLINE | 1      | 0           | 1000            | 0                   | 0       | 0              |         |
| 1            | 192.168.1.102 | 3306 | ONLINE | 1      | 0           | 1000            | 0                   | 0       | 0              |         |
| 1            | 192.168.1.103 | 3306 | ONLINE | 1      | 0           | 1000            | 0                   | 0       | 0              |         |
+--------------+---------------+------+--------+--------+-------------+-----------------+---------------------+---------+----------------+---------+
3 rows in set (0.00 sec)
所有操作都要记得load to runtime和save to disk

在mydb1库中执行
mysql> create user 'usr_proxysql_mon'@'192.168.1.%' identified by '2wsx3edc';
mysql> grant all privileges on *.* to 'usr_proxysql_mon'@'192.168.1.%';
mysql> flush privileges;

 

添加监控账号
(admin@127.0.0.1) [main]> set mysql-monitor_username='usr_proxysql_mon';
Query OK, 1 row affected (0.00 sec)
(admin@127.0.0.1) [main]> set mysql-monitor_password='2wsx3edc';
Query OK, 1 row affected (0.00 sec)
(admin@127.0.0.1) [main]> load mysql variables to runtime;
Query OK, 0 rows affected (0.00 sec)
(admin@127.0.0.1) [main]> save mysql variables to disk;
Query OK, 96 rows affected (0.30 sec)
(admin@127.0.0.1) [main]> select * from main.global_variables where variable_name in ('mysql-monitor_username','mysql-monitor_password');
+------------------------+------------------+
| variable_name          | variable_value   |
+------------------------+------------------+
| mysql-monitor_password | 2wsx3edc         |
| mysql-monitor_username | usr_proxysql_mon |
+------------------------+------------------+
2 rows in set (0.01 sec)
(admin@127.0.0.1) [main]> select * from main.runtime_global_variables where variable_name in ('mysql-monitor_username','mysql-monitor_password');
+------------------------+------------------+
| variable_name          | variable_value   |
+------------------------+------------------+
| mysql-monitor_password | 2wsx3edc         |
| mysql-monitor_username | usr_proxysql_mon |
+------------------------+------------------+
2 rows in set (0.00 sec)
(admin@127.0.0.1) [(none)]> select * from monitor.mysql_server_connect_log;

 

添加复制信息
(admin@127.0.0.1) [main]> insert into mysql_replication_hostgroups values (1,2,'cluster1');
Query OK, 1 row affected (0.00 sec)
(admin@127.0.0.1) [main]> select * from mysql_replication_hostgroups;
+------------------+------------------+----------+
| writer_hostgroup | reader_hostgroup | comment  |
+------------------+------------------+----------+
| 1                | 2                | cluster1 |
+------------------+------------------+----------+
1 row in set (0.00 sec)
(admin@127.0.0.1) [main]> select * from mysql_servers;
+--------------+---------------+------+--------+--------+-------------+-----------------+---------------------+---------+----------------+---------+
| hostgroup_id | hostname      | port | status | weight | compression | max_connections | max_replication_lag | use_ssl | max_latency_ms | comment |
+--------------+---------------+------+--------+--------+-------------+-----------------+---------------------+---------+----------------+---------+
| 1            | 192.168.1.101 | 3306 | ONLINE | 1      | 0           | 1000            | 0                   | 0       | 0              |         |
| 1            | 192.168.1.102 | 3306 | ONLINE | 1      | 0           | 1000            | 0                   | 0       | 0              |         |
| 1            | 192.168.1.103 | 3306 | ONLINE | 1      | 0           | 1000            | 0                   | 0       | 0              |         |
+--------------+---------------+------+--------+--------+-------------+-----------------+---------------------+---------+----------------+---------+
3 rows in set (0.01 sec)

(admin@127.0.0.1) [main]> load mysql servers to runtime;
Query OK, 0 rows affected (0.01 sec)
(admin@127.0.0.1) [main]> save mysql servers to disk;
Query OK, 0 rows affected (0.04 sec)
(admin@127.0.0.1) [main]>
(admin@127.0.0.1) [main]> select * from mysql_servers;
+--------------+---------------+------+--------+--------+-------------+-----------------+---------------------+---------+----------------+---------+
| hostgroup_id | hostname      | port | status | weight | compression | max_connections | max_replication_lag | use_ssl | max_latency_ms | comment |
+--------------+---------------+------+--------+--------+-------------+-----------------+---------------------+---------+----------------+---------+
| 1            | 192.168.1.101 | 3306 | ONLINE | 1      | 0           | 1000            | 0                   | 0       | 0              |         |
| 2            | 192.168.1.103 | 3306 | ONLINE | 1      | 0           | 1000            | 0                   | 0       | 0              |         |
| 2            | 192.168.1.102 | 3306 | ONLINE | 1      | 0           | 1000            | 0                   | 0       | 0              |         |
+--------------+---------------+------+--------+--------+-------------+-----------------+---------------------+---------+----------------+---------+
3 rows in set (0.00 sec)

 

在mydb1库中执行
mysql> create user 'root'@'192.168.1.%' identified by '1qaz2wsx';
mysql> grant all privileges on *.* to 'root'@'192.168.1.%';
mysql> create user 'msandbox'@'192.168.1.%' identified by '1qaz2wsx';
mysql> grant all privileges on *.* to 'msandbox'@'192.168.1.%';
mysql> flush privileges;

配置对外访问账号
(admin@127.0.0.1) [main]> select * from mysql_users;
Empty set (0.00 sec)
(admin@127.0.0.1) [main]> insert into main.mysql_users(username,password,default_hostgroup,transaction_persistent) values ('root','1qaz2wsx',1,1);
Query OK, 1 row affected (0.01 sec)
(admin@127.0.0.1) [main]> insert into main.mysql_users(username,password,default_hostgroup,transaction_persistent) values ('msandbox','1qaz2wsx',1,1);
Query OK, 1 row affected (0.00 sec)
(admin@127.0.0.1) [main]> select * from mysql_users;
+----------+----------+--------+---------+-------------------+----------------+---------------+------------------------+--------------+---------+----------+-----------------+
| username | password | active | use_ssl | default_hostgroup | default_schema | schema_locked | transaction_persistent | fast_forward | backend | frontend | max_connections |
+----------+----------+--------+---------+-------------------+----------------+---------------+------------------------+--------------+---------+----------+-----------------+
| root     | 1qaz2wsx | 1      | 0       | 1                 | NULL           | 0             | 1                      | 0            | 1       | 1        | 10000           |
| msandbox | 1qaz2wsx | 1      | 0       | 1                 | NULL           | 0             | 1                      | 0            | 1       | 1        | 10000           |
+----------+----------+--------+---------+-------------------+----------------+---------------+------------------------+--------------+---------+----------+-----------------+
2 rows in set (0.00 sec)
(admin@127.0.0.1) [main]> load mysql users to runtime;
Query OK, 0 rows affected (0.00 sec)
(admin@127.0.0.1) [main]> save mysql users to disk;
Query OK, 0 rows affected (0.00 sec)

# mysql -u root -p1qaz2wsx -h 127.0.0.1 -P6033
# mysql -u msandbox -p1qaz2wsx -h 127.0.0.1 -P6033

读写分离
配置读写分离策略需要使用mysql_query_rules表。表中的match_pattern字段就是代表设置的规则，destination_hostgroup字段代表默认指定的分组，apply代表真正执行应用规则
所有以select开头的语句全部分配到编号为2的读组中
select for update会产生一个写锁，对数据查询的时效性要求高，把它分配到编号为1的写组里，其他所有操作都会默认路由到写组中
(admin@127.0.0.1) [main]> select * from mysql_query_rules;
Empty set (0.00 sec)
(admin@127.0.0.1) [main]> insert into main.mysql_query_rules(active,match_pattern,destination_hostgroup, apply) VALUES(1,'^SELECT.*FOR UPDATE$',1,1);
Query OK, 1 row affected (0.00 sec)
(admin@127.0.0.1) [main]> insert into main.mysql_query_rules(active,match_pattern,destination_hostgroup, apply) VALUES(1,'^SELECT',2,1);
Query OK, 1 row affected (0.00 sec)
(admin@127.0.0.1) [main]> load mysql query rules to runtime;
Query OK, 0 rows affected (0.00 sec)
(admin@127.0.0.1) [main]> save mysql query rules to disk;
Query OK, 0 rows affected (0.05 sec)

(admin@127.0.0.1) [(none)]> select * from stats_mysql_query_digest;
(admin@127.0.0.1) [(none)]> select * from monitor.mysql_server_connect_log;

 

添加一个myuser:myuser的用户密码对
admin> select @@admin-admin_credentials;    # 当前用户名和密码
admin> set admin-admin_credentials='admin:admin;myuser:myuser';
admin> select @@admin-admin_credentials;
admin> load admin variables to runtime; 
admin> save admin variables to disk;

----------------------------------------------------------------------------------------------------------------------------------
通过sqldiff和keepalive及其状态监控机制来实现ProxySQL的故障转移并且在执行vip漂移前自动判断配置文件是否一致并同步
mydb1和mydb2连通性配置
ssh-keygen -t rsa
ssh-copy-id -i .ssh/id_rsa.pub root@192.168.1.101
ssh-copy-id -i .ssh/id_rsa.pub root@192.168.1.102

 

增加高可用能力：
Keepalived主要功能：
实现IP地址的漂移
执行健康检查
[root@mydb1 /]# wget http://www.keepalived.org/software/keepalived-1.4.5.tar.gz
[root@mydb1 /]# tar xvfz keepalived-1.4.5.tar.gz
[root@mydb1 /]# cd keepalived-1.4.5
[root@mydb1 keepalived-1.4.5]# ./configure --prefix=/usr/local/keepalived
[root@mydb1 keepalived-1.4.5]# make && make install
复制文件到相关路径，以方便调用
[root@mydb1 /]# cp /usr/local/keepalived/sbin/keepalived /usr/sbin/
[root@mydb1 ~]# vim /etc/init.d/keepalived
#!/bin/sh
#
# Startup script for the Keepalived daemon
#
# processname: keepalived
# pidfile: /var/run/keepalived.pid
# config: /etc/keepalived/keepalived.conf
# chkconfig: - 21 79
# description: Start and stop Keepalived

# Source function library
. /etc/rc.d/init.d/functions

# Source configuration file (we set KEEPALIVED_OPTIONS there)
. /etc/sysconfig/keepalived

RETVAL=0

prog="keepalived"

start() {
    echo -n $"Starting $prog: "
    daemon keepalived ${KEEPALIVED_OPTIONS}
    RETVAL=$?
    echo
    [ $RETVAL -eq 0 ] && touch /var/lock/subsys/$prog
}

stop() {
    echo -n $"Stopping $prog: "
    killproc keepalived
    RETVAL=$?
    echo
    [ $RETVAL -eq 0 ] && rm -f /var/lock/subsys/$prog
}

reload() {
    echo -n $"Reloading $prog: "
    killproc keepalived -1
    RETVAL=$?
    echo
}

# See how we were called.
case "$1" in
    start)
        start
        ;;
    stop)
        stop
        ;;
    reload)
        reload
        ;;
    restart)
        stop
        start
        ;;
    condrestart)
        if [ -f /var/lock/subsys/$prog ]; then
            stop
            start
        fi
        ;;
    status)
        status keepalived
        RETVAL=$?
        ;;
    *)
        echo "Usage: $0 {start|stop|reload|restart|condrestart|status}"
        RETVAL=1
esac

exit $RETVAL

[root@mydb1 ~]# chmod +x /etc/init.d/keepalived
[root@mydb1 /]# cp /usr/local/keepalived/etc/sysconfig/keepalived /etc/sysconfig/
默认情况下Keepalived会查找/etc/keepalived/keepalived.conf文件
[root@mydb1 /]# mkdir -p /etc/keepalived
[root@mydb1 /]# vim /etc/keepalived/keepalived.conf
!Configuration File for keepalived
global_defs {
 notification_email {
  allenhu@sina.com
    }
 notification_email_from allenhu@sina.com
     smtp_server 127.0.0.1
     smtp_connect_timeout 30
     router_id ProxySQL_HA
}
vrrp_script check_run {
 script "/usr/local/keepalived/check_proxysql.sh"
 interval 5
}
vrrp_instance proxysql {
 state BACKUP
                  nopreempt
 interface ens33
 virtual_router_id 51
 priority 100
 advert_int 1
 authentication {
  auth_type PASS
  auth_pass 3306
        }
 track_script {
  check_run
 }
 virtual_ipaddress {
  192.168.1.205/24
 }
}
[root@mydb1 /]# vim /usr/local/keepalived/check_proxysql.sh
#!/bin/sh
peer_ip='192.168.1.102'  #另一台ProxySQL的ip
peer_port=22           #ssh端口
proxysql='proxysql-1'  #本机的ProxySQL名字(自定义)
vip='192.168.1.205'      #keepalived的VIP
adm_email='allenhu@sina.com'
log=/usr/local/keepalived/proxysql.log
alias date='date +"%y-%m-%d_%H:%M:%S"'

#check if this keepalived is MASTER
ip a|grep $vip
if [ $? -ne 0 ];then  #如果当前不是主keepalived,则不需要检查
    exit 0
fi

#check if data port(6033) is alive
data_port_stats=$(timeout 2  bash -c 'cat < /dev/null > /dev/tcp/0.0.0.0/6033' &> /dev/null;echo $?)

if [ $data_port_stats -eq 0 ];then
    exit 0
else
    #check if the other keepalived is running
    peer_keepalived=$(ssh -p$peer_port $peer_ip 'systemctl is-active keepalived.service')
    if [ "$peer_keepalived" != "active" ];then
        echo "`date`  data port of $proxysql is not available, but the BACKUP keepalived is not running, so can't do the failover" >> $log
        echo "$proxysql is not available, but the BACKUP keepalived is not running. So please check immediately!
             "|/bin/mailx -S "from=proxysql@xxx.com <DoNotReply>" -s "proxysql error and can't failover" $adm_email
    else
        echo "`date`  data port of proxysql is not available, now SHUTDOWN keepalived." >> $log
        systemctl stop keepalived.service
    fi
fi

[root@mydb1 /]# vim /usr/local/keepalived/notify_master_proxysql.sh
#!/bin/sh
alias date='date +"%y-%m-%d_%H:%M:%S"'
peer_ip='192.168.1.102'
peer_port=22
proxysql='proxysql-1'
adm_email='allenhu@sina.com'
log=/usr/local/keepalived/proxysql.log

#when keepalived change to MASTER,first we should confirm the proxysql's config on this server is same with the previous proxysql
#thus we need no to deal with the proxysql's config between this two proxysql
rsync -a -e "ssh -p$peer_port" $peer_ip:/var/lib/proxysql/proxysql.db /tmp/proxysql.db
db_diff=$(/usr/local/sqlite-tools/sqldiff /var/lib/proxysql/proxysql.db /tmp/proxysql.db)
if [ $? -eq 0 ] && [ ! -z "$db_diff" ];then
    #sqldiff compare two sqlite file normally
    rsync -a /var/lib/proxysql/proxysql.db /var/lib/proxysql/proxysql.db_bak
    rsync -a /tmp/proxysql.db /var/lib/proxysql/proxysql.db
    echo "`date`  local config is different from the previous MASTER proxysql, already sync" >> $log
elif [ $? -ne 0 ];then
    #sqldiff cat't compare two sqlite file normally
    echo "`date`  can't compare config with previous MASTER proxysql, directly use local config" >> $log
fi

systemctl restart proxysql.service
if [ $? -eq 0 ];then
    echo "`date`  restart proxysql successful" >> $log
    add_msg="everything is ok now"
else
    echo "`date`  restart proxysql failed" >> $log
    add_msg="but can't restart proxysql, please check immediately!"
fi

echo "$proxysql's keepalived now change to MASTER, $add_msg"|/bin/mailx -S "from=proxysql@xxx.com <DoNotReply>" -s "$proxysql change to MASTER" $adm_email

[root@mydb1 /]# chmod +x /usr/local/keepalived/check_proxysql.sh
[root@mydb1 /]# chmod +x /usr/local/keepalived/notify_master_proxysql.sh
[root@mydb1 /]# /etc/init.d/keepalived start

[root@mydbl1 ~]# tail -100f /var/log/messages
Oct 14 00:21:38 mydb1 systemd: Started LVS and VRRP High Availability Monitor.
Oct 14 00:21:38 mydb1 Keepalived_healthcheckers[46918]: Opening file '/etc/keepalived/keepalived.conf'.
Oct 14 00:21:38 mydb1 Keepalived_vrrp[46919]: Registering Kernel netlink reflector
Oct 14 00:21:38 mydb1 Keepalived_vrrp[46919]: Registering Kernel netlink command channel
Oct 14 00:21:38 mydb1 Keepalived_vrrp[46919]: Registering gratuitous ARP shared channel
Oct 14 00:21:38 mydb1 Keepalived_vrrp[46919]: Opening file '/etc/keepalived/keepalived.conf'.
Oct 14 00:21:38 mydb1 Keepalived_vrrp[46919]: WARNING - default user 'keepalived_script' for script execution does not exist - please create.
Oct 14 00:21:38 mydb1 Keepalived_vrrp[46919]: SECURITY VIOLATION - scripts are being executed but script_security not enabled.
Oct 14 00:21:38 mydb1 Keepalived_vrrp[46919]: VRRP_Instance(proxysql) removing protocol VIPs.
Oct 14 00:21:38 mydb1 Keepalived_vrrp[46919]: Using LinkWatch kernel netlink reflector...
Oct 14 00:21:38 mydb1 Keepalived_vrrp[46919]: VRRP_Instance(proxysql) Entering BACKUP STATE
Oct 14 00:21:38 mydb1 Keepalived_vrrp[46919]: VRRP sockpool: [ifindex(2), proto(112), unicast(0), fd(10,11)]
Oct 14 00:21:38 mydb1 Keepalived_vrrp[46919]: VRRP_Script(check_run) succeeded
Oct 14 00:21:41 mydb1 Keepalived_vrrp[46919]: VRRP_Instance(proxysql) Transition to MASTER STATE
Oct 14 00:21:42 mydb1 Keepalived_vrrp[46919]: VRRP_Instance(proxysql) Entering MASTER STATE
Oct 14 00:21:42 mydb1 Keepalived_vrrp[46919]: VRRP_Instance(proxysql) setting protocol VIPs.
Oct 14 00:21:42 mydb1 Keepalived_vrrp[46919]: Sending gratuitous ARP on ens33 for 192.168.1.205
Oct 14 00:21:42 mydb1 Keepalived_vrrp[46919]: VRRP_Instance(proxysql) Sending/queueing gratuitous ARPs on ens33 for 192.168.1.205
Oct 14 00:21:42 mydb1 Keepalived_vrrp[46919]: Sending gratuitous ARP on ens33 for 192.168.1.205
Oct 14 00:21:42 mydb1 Keepalived_vrrp[46919]: Sending gratuitous ARP on ens33 for 192.168.1.205
Oct 14 00:21:42 mydb1 Keepalived_vrrp[46919]: Sending gratuitous ARP on ens33 for 192.168.1.205
Oct 14 00:21:42 mydb1 Keepalived_vrrp[46919]: Sending gratuitous ARP on ens33 for 192.168.1.205
Oct 14 00:21:42 mydb1 avahi-daemon[804]: Registering new address record for 192.168.1.205 on ens33.IPv4.
Oct 14 00:21:42 mydb1 systemd: Stopping LSB: High Performance Advanced Proxy for MySQL...
Oct 14 00:21:43 mydb1 proxysql: Shutting down ProxySQL: DONE!
Oct 14 00:21:43 mydb1 systemd: Starting LSB: High Performance Advanced Proxy for MySQL...
Oct 14 00:21:43 mydb1 proxysql: Starting ProxySQL: 2018-10-14 00:21:43 [INFO] Using config file /etc/proxysql.cnf
Oct 14 00:21:43 mydb1 proxysql: DONE!
Oct 14 00:21:43 mydb1 systemd: Started LSB: High Performance Advanced Proxy for MySQL.
Oct 14 00:21:47 mydb1 Keepalived_vrrp[46919]: Sending gratuitous ARP on ens33 for 192.168.1.205
Oct 14 00:21:47 mydb1 Keepalived_vrrp[46919]: VRRP_Instance(proxysql) Sending/queueing gratuitous ARPs on ens33 for 192.168.1.205
Oct 14 00:21:47 mydb1 Keepalived_vrrp[46919]: Sending gratuitous ARP on ens33 for 192.168.1.205
Oct 14 00:21:47 mydb1 Keepalived_vrrp[46919]: Sending gratuitous ARP on ens33 for 192.168.1.205
Oct 14 00:21:47 mydb1 Keepalived_vrrp[46919]: Sending gratuitous ARP on ens33 for 192.168.1.205
Oct 14 00:21:47 mydb1 Keepalived_vrrp[46919]: Sending gratuitous ARP on ens33 for 192.168.1.205

 

[root@mydb2 /]# wget http://www.keepalived.org/software/keepalived-1.4.5.tar.gz
[root@mydb2 /]# tar xvfz keepalived-1.4.5.tar.gz
[root@mydb2 /]# cd keepalived-1.4.5
[root@mydb2 keepalived-1.4.5]# ./configure --prefix=/usr/local/keepalived
[root@mydb2 keepalived-1.4.5]# make && make install
复制文件到相关路径，以方便调用
[root@mydb2 /]# cp /usr/local/keepalived/sbin/keepalived /usr/sbin/
[root@mydb2 ~]# vim /etc/init.d/keepalived
#!/bin/sh
#
# Startup script for the Keepalived daemon
#
# processname: keepalived
# pidfile: /var/run/keepalived.pid
# config: /etc/keepalived/keepalived.conf
# chkconfig: - 21 79
# description: Start and stop Keepalived

# Source function library
. /etc/rc.d/init.d/functions

# Source configuration file (we set KEEPALIVED_OPTIONS there)
. /etc/sysconfig/keepalived

RETVAL=0

prog="keepalived"

start() {
    echo -n $"Starting $prog: "
    daemon keepalived ${KEEPALIVED_OPTIONS}
    RETVAL=$?
    echo
    [ $RETVAL -eq 0 ] && touch /var/lock/subsys/$prog
}

stop() {
    echo -n $"Stopping $prog: "
    killproc keepalived
    RETVAL=$?
    echo
    [ $RETVAL -eq 0 ] && rm -f /var/lock/subsys/$prog
}

reload() {
    echo -n $"Reloading $prog: "
    killproc keepalived -1
    RETVAL=$?
    echo
}

# See how we were called.
case "$1" in
    start)
        start
        ;;
    stop)
        stop
        ;;
    reload)
        reload
        ;;
    restart)
        stop
        start
        ;;
    condrestart)
        if [ -f /var/lock/subsys/$prog ]; then
            stop
            start
        fi
        ;;
    status)
        status keepalived
        RETVAL=$?
        ;;
    *)
        echo "Usage: $0 {start|stop|reload|restart|condrestart|status}"
        RETVAL=1
esac

exit $RETVAL

[root@mydb2 ~]# chmod +x /etc/init.d/keepalived
[root@mydb2 /]# cp /usr/local/keepalived/etc/sysconfig/keepalived /etc/sysconfig/
默认情况下Keepalived会查找/etc/keepalived/keepalived.conf文件
[root@mydb2 /]# mkdir -p /etc/keepalived
[root@mydb2 /]# vim /etc/keepalived/keepalived.conf
!Configuration File for keepalived
global_defs {
 notification_email {
  allenhu@sina.com
    }
 notification_email_from allenhu@sina.com
     smtp_server 127.0.0.1
     smtp_connect_timeout 30
     router_id ProxySQL_HA
}
vrrp_script check_run {
 script "/usr/local/keepalived/check_proxysql.sh"
 interval 5
}
vrrp_instance proxysql {
 state BACKUP
                  nopreempt
 interface ens33
 virtual_router_id 51
 priority 100
 advert_int 1
 authentication {
  auth_type PASS
  auth_pass 3306
        }
 track_script {
  check_run
 }
 virtual_ipaddress {
  192.168.1.205/24
 }
}
[root@mydb2 /]# vim /usr/local/keepalived/check_proxysql.sh
#!/bin/sh
peer_ip='192.168.1.101'  #另一台ProxySQL的ip
peer_port=22           #ssh端口
proxysql='proxysql-2'  #本机的ProxySQL名字(自定义)
vip='192.168.1.205'      #keepalived的VIP
adm_email='allenhu@sina.com'
log=/usr/local/keepalived/proxysql.log
alias date='date +"%y-%m-%d_%H:%M:%S"'

#check if this keepalived is MASTER
ip a|grep $vip
if [ $? -ne 0 ];then  #如果当前不是主keepalived,则不需要检查
    exit 0
fi

#check if data port(6033) is alive
data_port_stats=$(timeout 2  bash -c 'cat < /dev/null > /dev/tcp/0.0.0.0/6033' &> /dev/null;echo $?)

if [ $data_port_stats -eq 0 ];then
    exit 0
else
    #check if the other keepalived is running
    peer_keepalived=$(ssh -p$peer_port $peer_ip 'systemctl is-active keepalived.service')
    if [ "$peer_keepalived" != "active" ];then
        echo "`date`  data port of $proxysql is not available, but the BACKUP keepalived is not running, so can't do the failover" >> $log
        echo "$proxysql is not available, but the BACKUP keepalived is not running. So please check immediately!
             "|/bin/mailx -S "from=proxysql@xxx.com <DoNotReply>" -s "proxysql error and can't failover" $adm_email
    else
        echo "`date`  data port of proxysql is not available, now SHUTDOWN keepalived." >> $log
        systemctl stop keepalived.service
    fi
fi

[root@mydb2 /]# vim /usr/local/keepalived/notify_master_proxysql.sh
#!/bin/sh
alias date='date +"%y-%m-%d_%H:%M:%S"'
peer_ip='192.168.1.101'
peer_port=22
proxysql='proxysql-2'
adm_email='allenhu@sina.com'
log=/usr/local/keepalived/proxysql.log

#when keepalived change to MASTER,first we should confirm the proxysql's config on this server is same with the previous proxysql
#thus we need no to deal with the proxysql's config between this two proxysql
rsync -a -e "ssh -p$peer_port" $peer_ip:/var/lib/proxysql/proxysql.db /tmp/proxysql.db
db_diff=$(/usr/local/sqlite-tools/sqldiff /var/lib/proxysql/proxysql.db /tmp/proxysql.db)
if [ $? -eq 0 ] && [ ! -z "$db_diff" ];then
    #sqldiff compare two sqlite file normally
    rsync -a /var/lib/proxysql/proxysql.db /var/lib/proxysql/proxysql.db_bak
    rsync -a /tmp/proxysql.db /var/lib/proxysql/proxysql.db
    echo "`date`  local config is different from the previous MASTER proxysql, already sync" >> $log
elif [ $? -ne 0 ];then
    #sqldiff cat't compare two sqlite file normally
    echo "`date`  can't compare config with previous MASTER proxysql, directly use local config" >> $log
fi

systemctl restart proxysql.service
if [ $? -eq 0 ];then
    echo "`date`  restart proxysql successful" >> $log
    add_msg="everything is ok now"
else
    echo "`date`  restart proxysql failed" >> $log
    add_msg="but can't restart proxysql, please check immediately!"
fi

echo "$proxysql's keepalived now change to MASTER, $add_msg"|/bin/mailx -S "from=proxysql@xxx.com <DoNotReply>" -s "$proxysql change to MASTER" $adm_email

[root@mydb2 /]# chmod +x /usr/local/keepalived/check_proxysql.sh
[root@mydb2 /]# chmod +x /usr/local/keepalived/notify_master_proxysql.sh
[root@mydb2 /]# /etc/init.d/keepalived start

[root@mydbl2 ~]# tail -100f /var/log/messages
Oct 14 00:36:30 mydb2 systemd: Starting LVS and VRRP High Availability Monitor...
Oct 14 00:36:30 mydb2 Keepalived[9349]: Starting Keepalived v1.4.5 (05/26,2018)
Oct 14 00:36:30 mydb2 Keepalived[9349]: Running on Linux 3.10.0-693.el7.x86_64 #1 SMP Tue Aug 22 21:09:27 UTC 2017 (built for Linux 3.10.0)
Oct 14 00:36:30 mydb2 Keepalived[9349]: Opening file '/etc/keepalived/keepalived.conf'.
Oct 14 00:36:30 mydb2 systemd: PID file /var/run/keepalived.pid not readable (yet?) after start.
Oct 14 00:36:30 mydb2 Keepalived[9350]: Starting Healthcheck child process, pid=9351
Oct 14 00:36:30 mydb2 Keepalived[9350]: Starting VRRP child process, pid=9352
Oct 14 00:36:30 mydb2 systemd: Started LVS and VRRP High Availability Monitor.
Oct 14 00:36:30 mydb2 Keepalived_healthcheckers[9351]: Opening file '/etc/keepalived/keepalived.conf'.
Oct 14 00:36:30 mydb2 Keepalived_vrrp[9352]: Registering Kernel netlink reflector
Oct 14 00:36:30 mydb2 Keepalived_vrrp[9352]: Registering Kernel netlink command channel
Oct 14 00:36:30 mydb2 Keepalived_vrrp[9352]: Registering gratuitous ARP shared channel
Oct 14 00:36:30 mydb2 Keepalived_vrrp[9352]: Opening file '/etc/keepalived/keepalived.conf'.
Oct 14 00:36:30 mydb2 Keepalived_vrrp[9352]: WARNING - default user 'keepalived_script' for script execution does not exist - please create.
Oct 14 00:36:30 mydb2 Keepalived_vrrp[9352]: SECURITY VIOLATION - scripts are being executed but script_security not enabled.
Oct 14 00:36:30 mydb2 Keepalived_vrrp[9352]: VRRP_Instance(proxysql) removing protocol VIPs.
Oct 14 00:36:30 mydb2 Keepalived_vrrp[9352]: Using LinkWatch kernel netlink reflector...
Oct 14 00:36:30 mydb2 Keepalived_vrrp[9352]: VRRP_Instance(proxysql) Entering BACKUP STATE
Oct 14 00:36:30 mydb2 Keepalived_vrrp[9352]: VRRP sockpool: [ifindex(2), proto(112), unicast(0), fd(10,11)]
Oct 14 00:36:30 mydb2 Keepalived_vrrp[9352]: VRRP_Script(check_run) succeeded

 

ProxySQL故障转移测试：
通过查看/var/log/messges日志，看出主备切换过程
在192.168.1.101上关闭ProxySQL服务，看VIP是否会切换到192.168.1.102上
开启192.168.1.101上的MySQL和keepalived，然后关闭192.168.1.102上的MySQL，看VIP是否会切换到192.168.1.101上