zoukankan      html  css  js  c++  java
  • mycat读写分离

    读写分离,简单的说就是把对数据库的读与写操作分开,以对应不同的数据库服务器。主数据库提供写操作,从数据库提供读操作,这样能有效地减轻单台数据库的压力。在很多系统中读操作的比例远高于写操作,所以对应的读操作的数据库可以有多台,通过负载均衡技术进一步分摊读操作的压力,让整个数据库系统高效,平稳地运行。而这些读写分离所需要的复杂数据库架构,要对开发人员和应用程序透明。这里我们通过mycat来实现上述的功能。

    mycat读写分离的控制由mycat的schema.xml配置文件中datahost标签的balance属性来实现。

    基于mysql的读写分离

    首先搭建MySQL的主从分离,因为这里只是测试mysql的读写分离,不再说明主从复制的搭建过程。主从搭建。

    主服务器:10.0.102.221
    从服务器:10.0.102.222
    备主服务器:10.0.102.179

    搭建过程: 在221和222之间搭建主从架构,在221和179之间搭建互为主从架构即可。也就是这三台服务器构成一个双主一从架构,主意和MySQL5.7的多源复制架构区分。
    多源复制

    搭建好测试需要的MySQL架构之后,然后开始配置mycat的读写分离。schema.xml中的主要部分配置文件如下:

            <schema name="TESTDB" checkSQLschema="false" sqlMaxLimit="100">
                    <!-- auto sharding by id (long) 
                    <table name="employee" dataNode="dn1,dn2,dn3" rule="sharding-by-murmur" /> -->
                    <table name="employee" dataNode="dn1"/>             #在这里没有分片操作,因此没有rule属性
            </schema>
            <!-- <dataNode name="dn1$0-743" dataHost="localhost1" database="db$0-743"
                    /> -->
            <dataNode name="dn1" dataHost="split-one" database="mytest" />            #指定dataNode
            <!--<dataNode name="dn4" dataHost="sequoiadb1" database="SAMPLE" />
            <dataNode name="jdbc_dn3"       dataHost="jdbchost" database="db3" /> -->
    
            <dataHost name="split-one" maxCon="1000" minCon="10" balance="0"
                              writeType="0" dbType="mysql" dbDriver="native" switchType="1"  slaveThreshold="100">
                    <heartbeat>select user()</heartbeat>
    
             #这个writeHost标签包含的一个主从的架构配置,w为master服务,r为slave服务器 <writeHost host="hostM1" url="10.0.102.221:3306" user="root" password="123456"> <readHost host="hostS2" url="10.0.102.222:3306" user="root" password="123456" /> </writeHost>
    #这个writehost标签包含的是一个备用的主服务器。 <writeHost host="hostM2" url="10.0.102.179:3306" user="root" password="123456" /> </dataHost>

    上面配置修改成功之后,启动mycat,登录到管理端口:

    mysql> show @@datasource;
    +----------+--------+-------+--------------+------+------+--------+------+------+---------+-----------+------------+
    | DATANODE | NAME   | TYPE  | HOST         | PORT | W/R  | ACTIVE | IDLE | SIZE | EXECUTE | READ_LOAD | WRITE_LOAD |
    +----------+--------+-------+--------------+------+------+--------+------+------+---------+-----------+------------+
    | dn1      | hostM1 | mysql | 10.0.102.221 | 3306 | W    |      0 |   10 | 1000 |      14 |         0 |          0 |
    | dn1      | hostM2 | mysql | 10.0.102.179 | 3306 | W    |      0 |    1 | 1000 |       4 |         0 |          0 |
    | dn1      | hostS2 | mysql | 10.0.102.222 | 3306 | R    |      0 |    3 | 1000 |       7 |         0 |          0 |
    +----------+--------+-------+--------------+------+------+--------+------+------+---------+-----------+------------+
    3 rows in set (0.01 sec)

    上面提到过,mycat的主从分离是有参数balance控制,下面根据不同的balance值测试对应读写分离。

    balance值为0时:不开启读写分离机制,所有的读操作都发送到当前可用的writeHost上。

    #插入数据,写入数据,发现这时候所有的读写都分配到当前可用的writeHost上,也就是主master上。
    mysql> show @@datasource; +----------+--------+-------+--------------+------+------+--------+------+------+---------+-----------+------------+ | DATANODE | NAME | TYPE | HOST | PORT | W/R | ACTIVE | IDLE | SIZE | EXECUTE | READ_LOAD | WRITE_LOAD | +----------+--------+-------+--------------+------+------+--------+------+------+---------+-----------+------------+ | dn1 | hostM1 | mysql | 10.0.102.221 | 3306 | W | 0 | 10 | 1000 | 94 | 11 | 1 | | dn1 | hostM2 | mysql | 10.0.102.179 | 3306 | W | 0 | 1 | 1000 | 72 | 0 | 0 | | dn1 | hostS2 | mysql | 10.0.102.222 | 3306 | R | 0 | 6 | 1000 | 78 | 0 | 0 | +----------+--------+-------+--------------+------+------+--------+------+------+---------+-----------+------------+ 3 rows in set (0.00 sec) mysql>

    balance值为1时全部的readHost与stand by writeHost都参与select语句的负载均衡。

    mysql> show @@datasource;
    +----------+--------+-------+--------------+------+------+--------+------+------+---------+-----------+------------+
    | DATANODE | NAME   | TYPE  | HOST         | PORT | W/R  | ACTIVE | IDLE | SIZE | EXECUTE | READ_LOAD | WRITE_LOAD |
    +----------+--------+-------+--------------+------+------+--------+------+------+---------+-----------+------------+
    | dn1      | hostM1 | mysql | 10.0.102.221 | 3306 | W    |      0 |   10 | 1000 |      19 |         0 |          1 |
    | dn1      | hostM2 | mysql | 10.0.102.179 | 3306 | W    |      0 |    1 | 1000 |      18 |        10 |          0 |
    | dn1      | hostS2 | mysql | 10.0.102.222 | 3306 | R    |      0 |    3 | 1000 |      16 |         5 |          0 |
    +----------+--------+-------+--------------+------+------+--------+------+------+---------+-----------+------------+
    3 rows in set (0.00 sec)

    #插入一条数据,是写在hostM1上,读的操作则分布在hostM2和hostS2上,测试select查询时,可以多查询几次。

    当balance值为2时所有的读操作都随机地在writeHost, readHost上分发。

    mysql> show @@datasource;
    +----------+--------+-------+--------------+------+------+--------+------+------+---------+-----------+------------+
    | DATANODE | NAME   | TYPE  | HOST         | PORT | W/R  | ACTIVE | IDLE | SIZE | EXECUTE | READ_LOAD | WRITE_LOAD |
    +----------+--------+-------+--------------+------+------+--------+------+------+---------+-----------+------------+
    | dn1      | hostM1 | mysql | 10.0.102.221 | 3306 | W    |      0 |   10 | 1000 |      20 |         6 |          0 |
    | dn1      | hostM2 | mysql | 10.0.102.179 | 3306 | W    |      0 |    1 | 1000 |       8 |         4 |          0 |
    | dn1      | hostS2 | mysql | 10.0.102.222 | 3306 | R    |      0 |    3 | 1000 |      14 |         7 |          0 |
    +----------+--------+-------+--------------+------+------+--------+------+------+---------+-----------+------------+
    3 rows in set (0.00 sec)

    #整个架构中所有的服务器都参与读的均衡,然后再插入数据,发现还是只有主master上写!
    mysql> show @@datasource;
    +----------+--------+-------+--------------+------+------+--------+------+------+---------+-----------+------------+
    | DATANODE | NAME   | TYPE  | HOST         | PORT | W/R  | ACTIVE | IDLE | SIZE | EXECUTE | READ_LOAD | WRITE_LOAD |
    +----------+--------+-------+--------------+------+------+--------+------+------+---------+-----------+------------+
    | dn1      | hostM1 | mysql | 10.0.102.221 | 3306 | W    |      0 |   10 | 1000 |      32 |         6 |          2 |
    | dn1      | hostM2 | mysql | 10.0.102.179 | 3306 | W    |      0 |    1 | 1000 |      18 |         4 |          0 |
    | dn1      | hostS2 | mysql | 10.0.102.222 | 3306 | R    |      0 |    3 | 1000 |      24 |         7 |          0 |
    +----------+--------+-------+--------------+------+------+--------+------+------+---------+-----------+------------+
    3 rows in set (0.01 sec)

    当balance值为3时:   所有的读请求都随机分配到writeHost对应的readHost上执行,writeHost不负担读压力,注意balance值为3只在mycat1.4版本及之后的版本中有。

    mysql> show @@datasource;
    +----------+--------+-------+--------------+------+------+--------+------+------+---------+-----------+------------+
    | DATANODE | NAME   | TYPE  | HOST         | PORT | W/R  | ACTIVE | IDLE | SIZE | EXECUTE | READ_LOAD | WRITE_LOAD |
    +----------+--------+-------+--------------+------+------+--------+------+------+---------+-----------+------------+
    | dn1      | hostM1 | mysql | 10.0.102.221 | 3306 | W    |      0 |   10 | 1000 |      14 |         0 |          0 |
    | dn1      | hostM2 | mysql | 10.0.102.179 | 3306 | W    |      0 |    1 | 1000 |       4 |         0 |          0 |
    | dn1      | hostS2 | mysql | 10.0.102.222 | 3306 | R    |      0 |    3 | 1000 |      17 |        10 |          0 |
    +----------+--------+-------+--------------+------+------+--------+------+------+---------+-----------+------------+
    3 rows in set (0.00 sec)

    #所有的查询都被分配到对应的slave上面。

    基于mariadb的galera集群做读写分离

    mariadb的galera集群时真正的多主架构,集群中的每台服务器都可以写入数据。首先搭建mariadb集群,过程略;搭建之后,配置mycat的读写分离。

    schema.xml配置文件如下,和之前的mysql主从架构配置会有一些不同。

            <schema name="TESTDB" checkSQLschema="false" sqlMaxLimit="100">
                    <!-- auto sharding by id (long) 
                    <table name="employee" dataNode="dn1,dn2,dn3" rule="sharding-by-murmur" /> -->
                    <table name="employee" dataNode="dn1"/>
            </schema>
            <!-- <dataNode name="dn1$0-743" dataHost="localhost1" database="db$0-743"
                    /> -->
            <dataNode name="dn1" dataHost="split-one" database="mytest" />
            <!--<dataNode name="dn4" dataHost="sequoiadb1" database="SAMPLE" />
            <dataNode name="jdbc_dn3"       dataHost="jdbchost" database="db3" /> -->
    
            <dataHost name="split-one" maxCon="1000" minCon="10" balance="0"
                              writeType="0" dbType="mysql" dbDriver="native" switchType="1"  slaveThreshold="100">
                    <heartbeat>show status like "wsrep%"</heartbeat>
    
                    <writeHost host="hostM1" url="10.0.102.204:3306" user="root"
                                       password="123456">
                    </writeHost>
    
                    <writeHost host="hostM2" url="10.0.102.221:3306" user="root"
                                       password="123456" >
                    </writeHost>
    
                    <writeHost host="hostM3" url="10.0.102.222:3306" user="root"
                                       password="123456" >
                    </writeHost>
            </dataHost>
    mysql> show @@datasource;
    +----------+--------+-------+--------------+------+------+--------+------+------+---------+-----------+------------+
    | DATANODE | NAME   | TYPE  | HOST         | PORT | W/R  | ACTIVE | IDLE | SIZE | EXECUTE | READ_LOAD | WRITE_LOAD |
    +----------+--------+-------+--------------+------+------+--------+------+------+---------+-----------+------------+
    | dn1      | hostM1 | mysql | 10.0.102.204 | 3306 | W    |      0 |    1 | 1000 |       1 |         0 |          0 |
    | dn1      | hostM2 | mysql | 10.0.102.221 | 3306 | W    |      0 |   10 | 1000 |      11 |         0 |          0 |
    | dn1      | hostM3 | mysql | 10.0.102.222 | 3306 | W    |      0 |    1 | 1000 |       1 |         0 |          0 |
    +----------+--------+-------+--------------+------+------+--------+------+------+---------+-----------+------------+
    3 rows in set (0.01 sec)
    
    mysql> show @@heartbeat;
    +--------+-------+--------------+------+---------+-------+--------+---------+--------------+---------------------+-------+
    | NAME   | TYPE  | HOST         | PORT | RS_CODE | RETRY | STATUS | TIMEOUT | EXECUTE_TIME | LAST_ACTIVE_TIME    | STOP  |
    +--------+-------+--------------+------+---------+-------+--------+---------+--------------+---------------------+-------+
    | hostM1 | mysql | 10.0.102.204 | 3306 |       1 |     0 | idle   |       0 | 19,19,19     | 2019-01-21 14:43:51 | false |
    | hostM2 | mysql | 10.0.102.221 | 3306 |       1 |     0 | idle   |       0 | 16,16,16     | 2019-01-21 14:43:51 | false |
    | hostM3 | mysql | 10.0.102.222 | 3306 |       1 |     0 | idle   |       0 | 17,17,17     | 2019-01-21 14:43:51 | false |
    +--------+-------+--------------+------+---------+-------+--------+---------+--------------+---------------------+-------+
    3 rows in set (0.00 sec)
    
    mysql> 

    mariadb的读写分离基本与上面mysql主从保持一致,不再重复测试。

    当balance的值为0时:不进行读写分离,所有的读写都走配置的dataHost的第一个writeHost
    当balance值为1时:全部的readHost和stand by writeHost都参与select语句的负载均衡。如果第一个节点未带slave节点,则三个节点中,一个作为写,另外两个作为读
    当balance为2时所有的读操作随机地在writeHost,readHost上分发。
    balance值为3时所有的读请求都随机地分发至writeHost对应的readHost执行,writeHost不负担读的压力。如果writeHost没有对应的readHost,则查询分发至writeHost。

    测试的过程中需要注意:一条insert语句插入多条记录,write_load的值只增加1.
    在插入数据测试的时候,可以采用如下方式查看,插入一个特殊的值,来确定数据究竟插入到了哪一个服务器中。

    mysql> insert into employee(id, name) values(4, @@HOSTNAME);
    Query OK, 1 row affected (0.04 sec)
    
    mysql> select * from employee where id =4;
    +------+-------+
    | id   | name  |
    +------+-------+
    |    4 | test2 |
    +------+-------+
    1 row in set (0.06 sec)
    
    mysql> 
  • 相关阅读:
    MIX11大会WP7主题演讲中文字幕版
    日本战神——源义经
    System.Web.HttpUtility for .Net Compact Framework
    VS2010 SP1
    <如何成为一个成功的职业经理人>读书笔记2
    <左手曾国藩,右手胡雪岩>读书笔记
    <福布斯荐75本经商必读06基业长青>读书笔记
    <中国人聪明之道>读书笔记
    <浮沉>读书笔记
    <79个潜规则:改变生活的心理学法则>读书笔记
  • 原文地址:https://www.cnblogs.com/wxzhe/p/10298731.html
Copyright © 2011-2022 走看看