zoukankan      html  css  js  c++  java
  • Mysql读写分离(mysql-proxy)

    MySQL-Proxy是一个处于你的client端和MySQL server端之间的简单程序,它可以监测、分析或改变它们的通信。它使用灵活,没有限制,常见的用途包括:负载平衡,故障、查询分析,查询过滤和修改等等。

    MySQL-Proxy就是这么一个中间层代理,简单的说,MySQL-Proxy就是一个连接池,负责将前台应用的连接请求转发给后台的数据库,并且通过使用lua脚本,可以实现复杂的连接控制和过滤,从而实现读写分离和负载平衡。对于应用来说,MySQL-Proxy是完全透明的,应用则只需要连接到MySQL-Proxy的监听端口即可。当然,这样proxy机器可能成为单点失效,但完全可以使用多个proxy机器做为冗余,在应用服务器的连接池配置中配置到多个proxy的连接参数即可。

    MySQL-Proxy更强大的一项功能是实现“读写分离”,基本原理是让主数据库处理事务性查询,让从库处理SELECT查询。数据库复制被用来把事务性查询导致的变更同步到集群中的从库。

    mysql-proxy 命令

    • --help-all ———— 用于获取全部帮助信息

    • --proxy-address=host:port ———— 代理服务监听的地址和端口

    • --admin-address=host:port ———— 管理模块监听的地址和端口

    • --proxy-backend-addresses=host:port ———— 后端mysql服务器的地址和端口(主服务器)

    • --proxy-read-only-backend-addresses=host:port ———— 后端只读mysql服务器的地址和端口(从服务器)

    • --proxy-lua-script=file ———— 完成mysql代理功能的Lua脚本

    • --daemon ———— 以守护进程模式启动mysql-proxy

    • --defaults-file=/path/to/conf_file_name ———— 默认使用的配置文件路径

    • --log-file=/path/to/log_file_name ———— 日志文件名称

    • --log-level=level ———— 日志级别

    • --log-use-syslog ———— 基于syslog记录日志

    • --user=user_name ———— 运行mysql-proxy进程的用户

    本文以下内容是在上一篇博客(mysql主从复制)基础上完成。

    http://www.cnblogs.com/Eivll0m/p/3776496.html

    1.环境及系统软件版本介绍:

    系统版本:CentOS 6.3_x86_64

    Mysql版本:mysql-5.6.17

    mysql-proxy版本:mysql-proxy-0.8.4(下载地址:http://cdn.mysql.com/Downloads/MySQL-Proxy/mysql-proxy-0.8.4-linux-glibc2.3-x86-64bit.tar.gz

    2.部署mysql-proxy,以下操作未经特殊说明,均表示在mysql-prxoy(172.16.10.60)服务器上进行。

    2.1检查是否安装了lua包,mysql-proxy需要用到lua脚本,以下表示已经安装过。

    # rpm -q lua
    lua-5.1.4-4.1.el6.x86_64

    2.2创建代理用户

    # useradd -r mysql-proxy

    2.3下载通用的压缩包mysql-proxy-0.8.4-linux-glibc2.3-x86-64bit.tar.gz(直接解压使用)

    # wget http://cdn.mysql.com/Downloads/MySQL-Proxy/mysql-proxy-0.8.4-linux-glibc2.3-x86-64bit.tar.gz
    # tar xf mysql-proxy-0.8.4-linux-glibc2.3-x86-64bit.tar.gz -C /usr/local
    # cd /usr/local/  
    # ln -sv mysql-proxy-0.8.3-linux-glibc2.3-x86-64bit mysql-proxy

    2.4设置mysql-proxy相关文件权限

    chown -R root:mysql-proxy /usr/local/mysql-proxy/*

    2.5更新环境变量

    # vi /etc/profile.d/mysql-proxy.sh
    export PATH=$PATH:/usr/local/mysql-proxy/bin
    # . /etc/profile

    2.6启动mysql-proxy

    # mysql-proxy  --daemon --log-level=debug --log-file=/var/log/mysql-proxy.log --plugins="proxy" --proxy-backend-addresses="172.16.10.72:3306" --proxy-read-only-backend-addresses="172.16.10.61:3306"

    2.7检查mysql-proxy是否正常启动

    通过命令netstat -antpl|grep mysql-prxoy或tail /var/log/mysql-proxy.log进行查看

    2.8在master服务器(172.16.10.72)上创建测试账号

    mysql> GRANT ALL ON *.* TO proxy@'172.16.10.%' IDENTIFIED BY '123456';        
    Query OK, 0 rows affected (0.22 sec)
    
    mysql> FLUSH PRIVILEGES;
    Query OK, 0 rows affected (0.03 sec)'

    2.9在slave服务器(172.16.10.61)上测试是否能通过代理端口4040 端口连接到 mysql-proxy(以下表示测试成功)

    复制代码
    root@mysql_slave ~]# mysql -uproxy -p -h172.16.10.60 -P 4040             
    Enter password: 
    Welcome to the MySQL monitor.  Commands end with ; or g.
    Your MySQL connection id is 456
    Server version: 5.6.17-log Source distribution
    
    Copyright (c) 2000, 2014, Oracle and/or its affiliates. All rights reserved.
    
    Oracle is a registered trademark of Oracle Corporation and/or its
    affiliates. Other names may be trademarks of their respective
    owners.
    
    Type 'help;' or 'h' for help. Type 'c' to clear the current input statement.
    
    mysql> 
    复制代码

    2.10配置读写分离,mysql-proxy 的安装目录中有一个名为rw-splitting.lua的 脚本,通过它可以实现读写分离,路径是/usr/local/mysql-proxy/share/doc/mysql-proxy/rw-splitting.lua。

    # ls /usr/local/mysql-proxy/share/doc/mysql-proxy/|grep rw-splitting.lua 
    rw-splitting.lua

    杀掉当前mysql-proxy进程,然后把lua脚本加入到启动选项里面重新启动mysql-proxy,实现读写分离

    # mysql-proxy --daemon --log-level=debug --log-file=/var/log/mysql-proxy.log --plugins="proxy" --proxy-backend-addresses="172.16.10.72:3306" --proxy-read-only-backend-addresses="172.16.10.61:3306" --proxy-lua-script=" /usr/local/mysql-proxy/share/doc/mysql-proxy/rw-splitting.lua"

    查看日志

    复制代码
    # tail /var/log/mysql-proxy.log 
    2014-06-13 16:23:04: (message) added read/write backend: 172.16.10.72:3306
    2014-06-13 16:23:04: (message) added read-only backend: 172.16.10.61:3306
    2014-06-13 16:40:11: (message) Initiating shutdown, requested from signal handler
    2014-06-13 16:40:11: (message) shutting down normally, exit code is: 0
    2014-06-13 16:40:17: (debug) chassis-path.c.122: adjusting relative path ( /usr/local/mysql-proxy/share/doc/mysql-proxy/rw-splitting.lua) to base_dir (/usr/local/mysql-proxy-0.8.4-linux-glibc2.3-x86-64bit). New path: /usr/local/mysql-proxy-0.8.4-linux-glibc2.3-x86-64bit/ /usr/local/mysql-proxy/share/doc/mysql-proxy/rw-splitting.lua
    2014-06-13 16:40:17: (critical) plugin proxy 0.8.4 started
    2014-06-13 16:40:17: (debug) max open file-descriptors = 1024
    2014-06-13 16:40:17: (message) proxy listening on port :4040
    2014-06-13 16:40:17: (message) added read/write backend: 172.16.10.72:3306
    2014-06-13 16:40:17: (message) added read-only backend: 172.16.10.61:3306
    复制代码

    2.11为mysql-proxy 提供一个管理接口,方便以后随时查看后端mysql 服务器的状态和访问类型,实现管理功能

     

    复制代码
    # vi /usr/local/mysql-proxy/share/doc/mysql-proxy/admin.lua
    function set_error(errmsg)  
        proxy.response = {  
            type = proxy.MYSQLD_PACKET_ERR,  
            errmsg = errmsg or "error"  
        }  
    end
    function read_query(packet) 
        if packet:byte() ~= proxy.COM_QUERY then  
            set_error("[admin] we only handle text-based queries (COM_QUERY)")  
            return proxy.PROXY_SEND_RESULT  
        end
        local query = packet:sub(2)
        local rows = { } 
        local fields = { }
        if query:lower() == "select * from backends" then 
            fields = {   
                { name = "backend_ndx",   
                  type = proxy.MYSQL_TYPE_LONG },
                { name = "address", 
                  type = proxy.MYSQL_TYPE_STRING },  
                { name = "state",  
                  type = proxy.MYSQL_TYPE_STRING },  
                { name = "type",  
                  type = proxy.MYSQL_TYPE_STRING },  
                { name = "uuid",  
                  type = proxy.MYSQL_TYPE_STRING },  
                { name = "connected_clients",   
                  type = proxy.MYSQL_TYPE_LONG },  
            }
            for i = 1, #proxy.global.backends do 
                local states = {  
                    "unknown",  
                    "up",  
                    "down"  
                }  
                local types = {  
                    "unknown",  
                    "rw",  
                    "ro"  
                }  
                local b = proxy.global.backends[i]
                rows[#rows + 1] = { 
                    i,  
                    b.dst.name,          -- configured backend address  
                    states[b.state + 1], -- the C-id is pushed down starting at 0  
                    types[b.type + 1],   -- the C-id is pushed down starting at 0  
                    b.uuid,              -- the MySQL Server's UUID if it is managed  
                    b.connected_clients  -- currently connected clients  
                }  
            end  
        elseif query:lower() == "select * from help" then  
            fields = {   
                { name = "command",   
                  type = proxy.MYSQL_TYPE_STRING },  
                { name = "description",   
                  type = proxy.MYSQL_TYPE_STRING },  
            }  
            rows[#rows + 1] = { "SELECT * FROM help", "shows this help" }  
            rows[#rows + 1] = { "SELECT * FROM backends", "lists the backends and their state" }  
        else  
            set_error("use 'SELECT * FROM help' to see the supported commands")  
            return proxy.PROXY_SEND_RESULT  
        end
        proxy.response = { 
            type = proxy.MYSQLD_PACKET_OK,  
            resultset = {  
                fields = fields,  
                rows = rows  
            }  
        }  
        return proxy.PROXY_SEND_RESULT  
    end
    复制代码

    2.12重新启动mysql-proxy

    注,这次启动要添加以下启动选项 ,因为我们添加了额外的插件,把新加功能添加进来,选项如下。

    • --plugins=admin  在mysql-proxy启动时加载的插件;

    • --admin-username="admin" 运行mysql-proxy进程管理的用户;

    • --admin-password="admin" 密码

    • --admin-lua-script="/usr/local/mysql-proxy/share/doc/mysql-proxy/admin.lua" 插件使用的配置文件路径;

    # killall mysql-proxy
    # mysql-proxy --daemon --log-level=debug --log-file=/var/log/mysql-proxy.log --plugins="proxy" --proxy-backend-addresses="172.16.10.72:3306" --proxy-read-only-backend-addresses="172.16.10.61:3306" --proxy-lua-script=" /usr/local/mysql-proxy/share/doc/mysql-proxy/rw-splitting.lua" --plugins=admin --admin-username="admin" --admin-password="admin" --admin-lua-script="/usr/local/mysql-proxy/share/doc/mysql-proxy/admin.lua"

    查看mysql-proxy是否启

    # netstat -antpl|grep mysql
    tcp        0      0 0.0.0.0:4040                0.0.0.0:*                   LISTEN      9511/mysql-proxy    
    tcp        0      0 0.0.0.0:4041                0.0.0.0:*                   LISTEN      9511/mysql-proxy   #4041为新生成的管理端口

    2.13在slave服务器(172.16.10.61)上进行测试,使用管理端口4041 登陆,密码是admin。(测试成功!)

    复制代码
    Bye
    [root@mysql_slave ~]# mysql -uadmin -p -h172.16.10.60 -P 4041                 
    Enter password: 
    Welcome to the MySQL monitor.  Commands end with ; or g.
    Your MySQL connection id is 1
    Server version: 5.0.99-agent-admin
    
    Copyright (c) 2000, 2014, Oracle and/or its affiliates. All rights reserved.
    
    Oracle is a registered trademark of Oracle Corporation and/or its
    affiliates. Other names may be trademarks of their respective
    owners.
    
    Type 'help;' or 'h' for help. Type 'c' to clear the current input statement.
    
    mysql> 
    复制代码

    2.14测试是否可以连接

    复制代码
    root@mysql_slave ~]# mysql -uadmin -p -h172.16.10.60 -P 4041                 
    Enter password: 
    Welcome to the MySQL monitor.  Commands end with ; or g.
    Your MySQL connection id is 1
    Server version: 5.0.99-agent-admin
    
    Copyright (c) 2000, 2014, Oracle and/or its affiliates. All rights reserved.
    
    Oracle is a registered trademark of Oracle Corporation and/or its
    affiliates. Other names may be trademarks of their respective
    owners.
    
    Type 'help;' or 'h' for help. Type 'c' to clear the current input statement.
    
    mysql> select * from backends;
    +-------------+-------------------+---------+------+------+-------------------+
    | backend_ndx | address           | state   | type | uuid | connected_clients |
    +-------------+-------------------+---------+------+------+-------------------+
    |           1 | 172.16.10.72:3306 | unknown | rw   | NULL |                 0 |
    |           2 | 172.16.10.61:3306 | unknown | ro   | NULL |                 0 |
    +-------------+-------------------+---------+------+------+-------------------+
    2 rows in set (0.00 sec)
    
    mysql> 
    复制代码

    由于此时没有进行任何操作,所以主从状态都是unknown。

    2.15配置启动脚本,并修连接端口(连接mysql-proxy时,原来用的端口是4040,下面修改为3306)

    复制代码
    # vi /etc/sysconfig/mysql-proxy
    # Options for mysql-proxy  
    ADMIN_USER="admin"  
    ADMIN_PASSWORD="admin"  
    ADMIN_ADDRESS=""  
    ADMIN_LUA_SCRIPT="/usr/local/mysql-proxy/share/doc/mysql-proxy/admin.lua"  
    PROXY_ADDRESS="0.0.0.0:3306"  
    PROXY_USER="mysql-proxy"  
    PROXY_OPTIONS="--daemon --log-level=info --log-file="/var/log/mysql-proxy.log" --plugins=proxy --plugins=admin --proxy-backend-addresses=172.16.10.72:3306 --proxy-read-only-backend-addresses=172.16.10.61:3306 --proxy-lua-script=/usr/local/mysql-proxy/share/doc/mysql-proxy/rw-splitting.lua"
    复制代码

    添加sysV脚本

    复制代码
    # vi /etc/init.d/mysql-proxy
    #!/bin/bash  
    #  
    # mysql-proxy This script starts and stops the mysql-proxy daemon  
    #  
    # chkconfig: - 78 30  
    # processname: mysql-proxy  
    # description: mysql-proxy is a proxy daemon for mysql
    # Source function library. 
    . /etc/rc.d/init.d/functions
    prog="/usr/local/mysql-proxy/bin/mysql-proxy"
    # Source networking configuration. 
    if [ -f /etc/sysconfig/network ]; then  
        . /etc/sysconfig/network  
    fi
    # Check that networking is up. 
    [ ${NETWORKING} = "no" ] && exit 0
    # Set default mysql-proxy configuration. 
    ADMIN_USER="admin"  
    ADMIN_PASSWD="admin"  
    ADMIN_LUA_SCRIPT="/usr/local/mysql-proxy/share/doc/mysql-proxy/admin.lua"  
    PROXY_OPTIONS="--daemon"  
    PROXY_PID=/var/run/mysql-proxy.pid  
    PROXY_USER="mysql-proxy"
    # Source mysql-proxy configuration. 
    if [ -f /etc/sysconfig/mysql-proxy ]; then  
        . /etc/sysconfig/mysql-proxy  
    fi
    RETVAL=0
    start() { 
        echo -n $"Starting $prog: "  
        daemon $prog $PROXY_OPTIONS --pid-file=$PROXY_PID --proxy-address="$PROXY_ADDRESS" --user=$PROXY_USER --admin-username="$ADMIN_USER" --admin-lua-script="$ADMIN_LUA_SCRIPT" --admin-password="$ADMIN_PASSWORD"  
        RETVAL=$?  
        echo  
        if [ $RETVAL -eq 0 ]; then  
            touch /var/lock/subsys/mysql-proxy  
        fi  
    }
    stop() { 
        echo -n $"Stopping $prog: "  
        killproc -p $PROXY_PID -d 3 $prog  
        RETVAL=$?  
        echo  
        if [ $RETVAL -eq 0 ]; then  
            rm -f /var/lock/subsys/mysql-proxy  
            rm -f $PROXY_PID  
        fi  
    }  
    # See how we were called.  
    case "$1" in  
        start)  
            start  
            ;;  
        stop)  
            stop  
            ;;  
        restart)  
            stop  
            start  
            ;;  
        condrestart|try-restart)  
            if status -p $PROXY_PIDFILE $prog >&/dev/null; then  
                stop  
                start  
            fi  
            ;;  
        status)  
            status -p $PROXY_PID $prog  
            ;;  
        *)  
            echo "Usage: $0 {start|stop|restart|reload|status|condrestart|try-restart}"  
            RETVAL=1  
            ;;  
    esac
    exit $RETVAL
    复制代码
    # chmod +x /etc/rc.d/init.d/mysql-proxy
    # chkconfig --add mysql-proxy
    # chkconfig mysql-proxy on

    2.16重启mysql-proxy并查看端口(4040端口已消失)

    # service mysql-proxy restart
    # netstat -antpl|grep mysql      
    tcp        0      0 0.0.0.0:4041                0.0.0.0:*                   LISTEN      12427/mysql-proxy   
    tcp        0      0 0.0.0.0:3306                0.0.0.0:*                   LISTEN      12427/mysql-proxy  

    2.17测试读写分离

    执行读写操作

    复制代码
    [root@mysql_slave ~]# mysql -uproxy -p -h172.16.10.60 -e "show databases;"     
    Enter password: 
    +--------------------+
    | Database           |
    +--------------------+
    | information_schema |
    | eivll0m            |
    | mysql              |
    | performance_schema |
    | test               |
    +--------------------+
    [root@mysql_slave ~]# mysql -uproxy -p -h172.16.10.60 -e "create database mydb1;"
    Enter password: 
    复制代码
    复制代码
    mysql> select * from backends;
    +-------------+-------------------+---------+------+------+-------------------+
    | backend_ndx | address           | state   | type | uuid | connected_clients |
    +-------------+-------------------+---------+------+------+-------------------+
    |           1 | 172.16.10.72:3306 | up      | rw   | NULL |                 0 |
    |           2 | 172.16.10.61:3306 | unknown | ro   | NULL |                 0 |
    +-------------+-------------------+---------+------+------+-------------------+
    2 rows in set (0.00 sec)
    复制代码

    由于master是支持读写的,所以只发现master状态变为了up是正常现象,要想观看到从的状态,可以执行多次查询。

    复制代码
    # mysql -uproxy -p -h172.16.10.60 -e "select user,host from mysql.user"
    Enter password: 
    +-------+--------------+
    | user  | host         |
    +-------+--------------+
    | root  | 127.0.0.1    |
    | proxy | 172.16.10.%  |
    | root  | ::1          |
    |       | localhost    |
    | root  | localhost    |
    |       | mysql\_slave |
    | root  | mysql\_slave |
    +-------+--------------+
    
    mysql> select * from backends;
    +-------------+-------------------+-------+------+------+-------------------+
    | backend_ndx | address           | state | type | uuid | connected_clients |
    +-------------+-------------------+-------+------+------+-------------------+
    |           1 | 172.16.10.72:3306 | up    | rw   | NULL |                 0 |
    |           2 | 172.16.10.61:3306 | up    | ro   | NULL |                 0 |
    +-------------+-------------------+-------+------+------+-------------------+
    2 rows in set (0.00 sec)
    复制代码

    现在,主从状态全部变成了up状态。

     

  • 相关阅读:
    关于两端对齐
    关于删除节点的兼容写法
    因为一个css,导致网页在手机上滑动不流畅
    Java垃圾回收机制
    Java的ArrayList
    大神说,规则引擎,反正不懂,留个纪念,以后看
    Java克隆
    i++和++i的区别
    JavaScript中的方法或者变量名称前面有下划线,是做什么的?
    RandomAccess接口是空的,那它是用来做什么的呢?
  • 原文地址:https://www.cnblogs.com/hllnj2008/p/3993293.html
Copyright © 2011-2022 走看看