zoukankan      html  css  js  c++  java
  • MyCAT部署及实现读写分离(转)

    MyCAT是mysql中间件,前身是阿里大名鼎鼎的Cobar,Cobar在开源了一段时间后,不了了之。于是MyCAT扛起了这面大旗,在大数据时代,其重要性愈发彰显。这篇文章主要是MyCAT的入门部署。

    一、安装java

    因Mycat是用java开发的,所以需要在实验环境下安装java,官方建议jdk1.7及以上版本

    Java Oracle官方下载地址为:

    http://www.oracle.com/technetwork/java/javase/archive-139210.html

    解压jdk-7u79-linux-x64.tar.gz文件,并配置java的环境变量

    # tar xvf jdk-7u79-linux-x64.tar.gz

    # mv jdk1.7.0_79/ /usr/local/

    编辑/etc/profile文件

    # vim /etc/profile  添加如下内容

    export JAVA_HOME=/usr/local/jdk1.7.0_79
    export PATH=$JAVA_HOME/bin:$PATH
    export CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar

    # source /etc/profile --使/etc/profile文件生效

    # java -version

    java version "1.7.0_79"
    Java(TM) SE Runtime Environment (build 1.7.0_79-b15)
    Java HotSpot(TM) 64-Bit Server VM (build 24.79-b02, mixed mode)

    二、安装MySQL

    MySQL官方下载地址如下:

    http://dev.mysql.com/downloads/mysql/

    里面有各种版本,包括RPM,二进制,源码包。

    为了方便起见,我这里选择是RPM包

    # yum install -y mysql-community-server-5.6.26-2.el5.x86_64.rpm

    启动MySQL

    # /etc/init.d/mysqld start

    Starting mysqld (via systemctl):                           [  OK  ]

    为root账户创建密码

    # mysqladmin -u root password "123456"

    建议为123456,后面MyCAT配置文件schema.xml中会用到

    三、安装MyCAT

    创建MyCAT用户并设置密码

    # useradd mycat

    # passwd mycat

    解压MyCAT文件

    # tar xvf Mycat-server-1.5-alpha-20151221110028-linux.tar.gz

    # mv mycat/ /usr/local/

    设置mycat目录的属主和属组

    # chown -R mycat.mycat /usr/local/mycat/

    四、测试MyCAT

    首先在MySQL上创建三个数据库:db1,db2,db3。

    复制代码
    mysql> create database db1;
    Query OK, 1 row affected (0.00 sec)
    
    mysql> create database db2;
    Query OK, 1 row affected (0.00 sec)
    
    mysql> create database db3;
    Query OK, 1 row affected (0.00 sec)
    复制代码

    启动mycat服务

    # cd /usr/local/mycat/bin/

    # ./mycat start

    Starting Mycat-server...

    查看mycat服务是否启动

    # ps -ef |grep mycat

    root       9640   7257  0 22:47 pts/3    00:00:00 grep --color=auto mycat

    并没有启动

    查看日志

    # cd /usr/local/mycat/logs/

    # cat wrapper.log

    复制代码
    STATUS | wrapper  | 2016/01/07 22:44:23 | --> Wrapper Started as Daemon
    STATUS | wrapper  | 2016/01/07 22:44:23 | Launching a JVM...
    ERROR  | wrapper  | 2016/01/07 22:44:25 | JVM exited while loading the application.
    INFO   | jvm 1    | 2016/01/07 22:44:25 | Error: Exception thrown by the agent : java.net.MalformedURLException: Local host name unknown: java.net.UnknownHostException: MySQL-Server1: MySQL-Server1: Name or service not known
    STATUS | wrapper  | 2016/01/07 22:44:29 | Launching a JVM...
    ERROR  | wrapper  | 2016/01/07 22:44:29 | JVM exited while loading the application.
    INFO   | jvm 2    | 2016/01/07 22:44:29 | Error: Exception thrown by the agent : java.net.MalformedURLException: Local host name unknown: java.net.UnknownHostException: MySQL-Server1: MySQL-Server1: Name or service not known
    STATUS | wrapper  | 2016/01/07 22:44:34 | Launching a JVM...
    ERROR  | wrapper  | 2016/01/07 22:44:34 | JVM exited while loading the application.
    INFO   | jvm 3    | 2016/01/07 22:44:34 | Error: Exception thrown by the agent : java.net.MalformedURLException: Local host name unknown: java.net.UnknownHostException: MySQL-Server1: MySQL-Server1: Name or service not known
    STATUS | wrapper  | 2016/01/07 22:44:38 | Launching a JVM...
    ERROR  | wrapper  | 2016/01/07 22:44:38 | JVM exited while loading the application.
    INFO   | jvm 4    | 2016/01/07 22:44:38 | Error: Exception thrown by the agent : java.net.MalformedURLException: Local host name unknown: java.net.UnknownHostException: MySQL-Server1: MySQL-Server1: Name or service not known
    STATUS | wrapper  | 2016/01/07 22:44:42 | Launching a JVM...
    ERROR  | wrapper  | 2016/01/07 22:44:43 | JVM exited while loading the application.
    INFO   | jvm 5    | 2016/01/07 22:44:43 | Error: Exception thrown by the agent : java.net.MalformedURLException: Local host name unknown: java.net.UnknownHostException: MySQL-Server1: MySQL-Server1: Name or service not known
    FATAL  | wrapper  | 2016/01/07 22:44:43 | There were 5 failed launches in a row, each lasting less than 300 seconds.  Giving up.
    FATAL  | wrapper  | 2016/01/07 22:44:43 |   There may be a configuration problem: please check the logs.
    STATUS | wrapper  | 2016/01/07 22:44:43 | <-- Wrapper Stopped
    复制代码

    根据报错信息,怀疑主机名没有绑定

    修改hosts文件,绑定主机名

    # vim /etc/hosts

    127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
    ::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
    192.168.244.144 MySQL-Server1

    重新启动mycat服务

    # cd /usr/local/mycat/bin/

    # ./mycat start

    这次mycat正常启动

    # ps -ef |grep mycat

    root      10725      1  0 22:54 ?        00:00:00 /usr/local/mycat/bin/./wrapper-linux-x86-64 /usr/local/mycat/conf/wrapper.conf wrapper.syslog.ident=mycat wrapper.pidfile=/usr/local/mycat/logs/mycat.pid wrapper.daemonize=TRUE wrapper.lockfile=/var/lock/subsys/mycat
    root      10881   7257  0 22:55 pts/3    00:00:00 grep --color=auto mycat

    下面以Travelrecord为例,来进行插入,查询,路由分析等基本操作。

    首先利用mycat连接mysql数据库

    # mysql -utest -ptest -h127.0.0.1 -P8066 -DTESTDB

    其中8066是mycat的监听端口,类似于mysql的3306端口,其中-u,-p,-h分别是用户名,密码和主机,-D是连接的逻辑库。

    至于为什么是这些,这个跟配置文件有关。

    红色部分表明连接的是mycat。

    创建Travelrecord表

    create table travelrecord (id bigint not null primary key,user_id varchar(100),traveldate DATE, fee decimal,days int);

    插入数据

    复制代码
    mysql> insert into travelrecord(id,user_id,traveldate,fee,days)  values(1,'Victor',20160101,100,10);
    Query OK, 1 row affected (0.00 sec)
    
    mysql> insert into travelrecord(id,user_id,traveldate,fee,days)  values(5000001,'Job',20160102,100,10);
    Query OK, 1 row affected (0.00 sec)
    
    mysql> insert into travelrecord(id,user_id,traveldate,fee,days)  values(10000001,'Slow',20160103,100,10);
    Query OK, 1 row affected (0.00 sec)
    复制代码

    至于ID为什么取三个值,这个与conf目录下autopartition-long.txt的定义有关,这个文件主要定义auto-sharding-long的规则。

    # range start-end ,data node index
    # K=1000,M=10000.
    0-500M=0
    500M-1000M=1
    1000M-1500M=2

    我主要是测试在id取不同区间的值时,分片的效果。

    下面来看看分片的效果

    复制代码
    mysql> select * from db1.travelrecord;
    +----+---------+------------+------+------+
    | id | user_id | traveldate | fee  | days |
    +----+---------+------------+------+------+
    |  1 | Victor  | 2016-01-01 |  100 |   10 |
    +----+---------+------------+------+------+
    1 row in set (0.00 sec)
    
    mysql> select * from db2.travelrecord;
    +---------+---------+------------+------+------+
    | id      | user_id | traveldate | fee  | days |
    +---------+---------+------------+------+------+
    | 5000001 | Job     | 2016-01-02 |  100 |   10 |
    +---------+---------+------------+------+------+
    1 row in set (0.00 sec)
    
    mysql> select * from db3.travelrecord;
    +----------+---------+------------+------+------+
    | id       | user_id | traveldate | fee  | days |
    +----------+---------+------------+------+------+
    | 10000001 | Slow    | 2016-01-03 |  100 |   10 |
    +----------+---------+------------+------+------+
    1 row in set (0.00 sec)
    复制代码

    果然是分配到三个不同的节点上了。

    如果想看MyCAT具体会将数据分配到哪个节点上,可通过路由分析。

    语法其实蛮简单,就是SQL语句前加上explain语句。

    下面根据explain命令查看create语句和insert语句具体会分配到哪些Datanode上。

    复制代码
    mysql> explain create table travelrecord (id bigint not null primary key,user_id varchar(100),traveldate DATE, fee decimal,days int);
    +-----------+-----------------------------------------------------------------------------------------------------------------------+
    | DATA_NODE | SQL                                                                                                                   |
    +-----------+-----------------------------------------------------------------------------------------------------------------------+
    | dn1       | create table travelrecord (id bigint not null primary key,user_id varchar(100),traveldate DATE, fee decimal,days int) |
    | dn2       | create table travelrecord (id bigint not null primary key,user_id varchar(100),traveldate DATE, fee decimal,days int) |
    | dn3       | create table travelrecord (id bigint not null primary key,user_id varchar(100),traveldate DATE, fee decimal,days int) |
    +-----------+-----------------------------------------------------------------------------------------------------------------------+
    3 rows in set (0.00 sec)
    
    mysql>  explain insert into travelrecord(id,user_id,traveldate,fee,days)  values(1,'Victor',20160101,100,10);
    +-----------+----------------------------------------------------------------------------------------------+
    | DATA_NODE | SQL                                                                                          |
    +-----------+----------------------------------------------------------------------------------------------+
    | dn1       | insert into travelrecord(id,user_id,traveldate,fee,days)  values(1,'Victor',20160101,100,10) |
    +-----------+----------------------------------------------------------------------------------------------+
    1 row in set (0.01 sec)
    
    mysql> explain insert into travelrecord(id,user_id,traveldate,fee,days)  values(5000001,'Job',20160102,100,10);
    +-----------+-------------------------------------------------------------------------------------------------+
    | DATA_NODE | SQL                                                                                             |
    +-----------+-------------------------------------------------------------------------------------------------+
    | dn2       | insert into travelrecord(id,user_id,traveldate,fee,days)  values(5000001,'Job',20160102,100,10) |
    +-----------+-------------------------------------------------------------------------------------------------+
    1 row in set (0.00 sec)
    
    mysql> explain insert into travelrecord(id,user_id,traveldate,fee,days)  values(10000001,'Slow',20160103,100,10);
    +-----------+---------------------------------------------------------------------------------------------------+
    | DATA_NODE | SQL                                                                                               |
    +-----------+---------------------------------------------------------------------------------------------------+
    | dn3       | insert into travelrecord(id,user_id,traveldate,fee,days)  values(10000001,'Slow',20160103,100,10) |
    +-----------+---------------------------------------------------------------------------------------------------+
    1 row in set (0.00 sec)
    复制代码

    总结:

    关于MyCAT的配置其实是蛮简单的,最主要的是熟悉各配置文件的规则。以上用户名,密码,如何分库,都是在配置文件中定义的,后续,有时间再一一详表。

    关于配置文件,conf目录下主要以下三个需要熟悉。

    server.xml是Mycat服务器参数调整和用户授权的配置文件

    schema.xml是逻辑库定义和表以及分片定义的配置文件

    rule.xml是分片规则的配置文件

    http://www.cnblogs.com/ivictor/p/5111495.html
     

    在MySQL中间件出现之前,对于MySQL主从集群,如果要实现其读写分离,一般是在程序端实现,这样就带来一个问题,即数据库和程序的耦合度太高,如果我数据库的地址发生改变了,那么我程序端也要进行相应的修改,如果数据库不小心挂掉了,则同时也意味着程序的不可用,而这对很多应用来说,并不能接受。

    引入MySQL中间件能很好的对程序端和数据库进行解耦,这样,程序端只需关注数据库中间件的地址,而无需知晓底层数据库是如何提供服务。

    作为当前炙手可热的MySQL中间件,MyCAT实现MySQL主从集群的读写分离自是应有之义,其配置也相当简单。

    在这里,我用三个实例组成MySQL主从集群,来验证MyCAT的读写分离功能,其实,一主一从就可以满足,之所以用三个,是为了验证MyCAT的分片功能。

    集群组成如下:

    角色             主机名                      主机IP

    master         mysql-server1          192.168.244.145

    slave            mysql-server2          192.168.244.146

    slave            mysql-server3          192.168.244.144

    在这里,还是使用Travelrecord表进行测试。

    首先编辑MyCAT的配置文件schema.xml,关于dataHost的配置信息如下:

    复制代码
    <dataHost name="localhost1" maxCon="1000" minCon="10" balance="1"
                    writeType="0" dbType="mysql" dbDriver="native" switchType="-1"  slaveThreshold="100">
                    <heartbeat>select user()</heartbeat>
                    <!-- can have multi write hosts -->
                    <writeHost host="hostM1" url="localhost:3306" user="root"
                            password="123456">
                    </writeHost>
                    <writeHost host="hostS1" url="192.168.244.146:3306" user="root"
                            password="123456" />
                    <writeHost host="hostS2" url="192.168.244.144:3306" user="root"                      
                            password="123456" />
    </dataHost>
    复制代码

    这里面,有两个参数需要注意,balance和 switchType。

    其中,balance指的负载均衡类型,目前的取值有4种:

    1. balance="0", 不开启读写分离机制,所有读操作都发送到当前可用的writeHost上。

    2. balance="1",全部的readHost与stand by writeHost参与select语句的负载均衡,简单的说,当双主双从模式(M1->S1,M2->S2,并且M1与 M2互为主备),正常情况下,M2,S1,S2都参与select语句的负载均衡。

    3. balance="2",所有读操作都随机的在writeHost、readhost上分发。

    4. balance="3",所有读请求随机的分发到wiriterHost对应的readhost执行,writerHost不负担读压力

    switchType指的是切换的模式,目前的取值也有4种:

    1. switchType='-1' 表示不自动切换

    2. switchType='1' 默认值,表示自动切换

    3. switchType='2' 基于MySQL主从同步的状态决定是否切换,心跳语句为 show slave status

    4. switchType='3'基于MySQL galary cluster的切换机制(适合集群)(1.4.1),心跳语句为 show status like 'wsrep%'。

    因此,该配置文件中的balance="1"意味着作为stand by writeHost的hostS1和hostS2将参与select语句的负载均衡,这就实现了主从的读写分离,switchType='-1'意味着当主挂掉的时候,不进行自动切换,即hostS1和hostS2并不会被提升为主,仍只提供读的功能。这就避免了将数据读进slave的可能性,毕竟,单纯的MySQL主从集群并不允许将数据读进slave中,除非配置的是双master。

    验证读写分离

    下面来验证一下,

    创建Travelrecord表

    create table travelrecord (id bigint not null primary key,user_id varchar(100),traveldate DATE, fee decimal,days int);

    插入数据

    mysql> insert into travelrecord(id,user_id,traveldate,fee,days)  values(1,@@hostname,20160101,100,10);
    Query OK, 1 row affected, 1 warning (0.02 sec)
    
    mysql> insert into travelrecord(id,user_id,traveldate,fee,days)  values(5000001,@@hostname,20160102,100,10);
    Query OK, 1 row affected, 1 warning (0.01 sec)

    在这里,用了一个取巧的方法,即对user_id插入了当前实例的主机名,这样可直观的观察读写是否分离以及MyCAT的分片功能。能这样做的原因在于我当前的MySQL版本-5.6.26默认是基于statement的复制,如果是基于row的复制,则这个方法将不可取。

    查询数据

    复制代码
    mysql> select * from travelrecord;
    +---------+---------------+------------+------+------+
    | id      | user_id       | traveldate | fee  | days |
    +---------+---------------+------------+------+------+
    |       1 | mysql-server2 | 2016-01-01 |  100 |   10 |
    | 5000001 | mysql-server3 | 2016-01-02 |  100 |   10 |
    +---------+---------------+------------+------+------+
    2 rows in set (0.01 sec)
    
    mysql> select * from travelrecord;
    +---------+---------------+------------+------+------+
    | id      | user_id       | traveldate | fee  | days |
    +---------+---------------+------------+------+------+
    | 5000001 | mysql-server3 | 2016-01-02 |  100 |   10 |
    |       1 | mysql-server2 | 2016-01-01 |  100 |   10 |
    +---------+---------------+------------+------+------+
    2 rows in set (0.02 sec)
    
    mysql> select * from travelrecord;
    +---------+---------------+------------+------+------+
    | id      | user_id       | traveldate | fee  | days |
    +---------+---------------+------------+------+------+
    | 5000001 | mysql-server3 | 2016-01-02 |  100 |   10 |
    |       1 | mysql-server3 | 2016-01-01 |  100 |   10 |
    +---------+---------------+------------+------+------+
    2 rows in set (0.01 sec)
    
    mysql> select * from travelrecord;
    +---------+---------------+------------+------+------+
    | id      | user_id       | traveldate | fee  | days |
    +---------+---------------+------------+------+------+
    | 5000001 | mysql-server3 | 2016-01-02 |  100 |   10 |
    |       1 | mysql-server3 | 2016-01-01 |  100 |   10 |
    +---------+---------------+------------+------+------+
    2 rows in set (0.01 sec)
    
    mysql> select * from travelrecord;
    +---------+---------------+------------+------+------+
    | id      | user_id       | traveldate | fee  | days |
    +---------+---------------+------------+------+------+
    |       1 | mysql-server2 | 2016-01-01 |  100 |   10 |
    | 5000001 | mysql-server2 | 2016-01-02 |  100 |   10 |
    +---------+---------------+------------+------+------+
    复制代码

    从上面的输出结果,可以得出以下两点:

    一、该配置已实现读写分离,读出来的数据没有master节点的。

    二、MyCAT的随机分发不是基于statement的,即一个select语句查询其中一个节点,另外一个select语句查询另外一个节点。它分发针对的是片的,同一个select语句的结果是有不同dataNode返回的。

    不仅如此,从MyCAT日志中也可以获取读写分离的相关信息,当然,前提是MyCAT的日志级别是debug。日志相关信息如下:

    验证mater挂了,slave还能提供读的功能

    对于MySQL主从集群,我们的需求是master挂了,slave还能提供读的功能。

    下面来测试一下

    首先,人为的关闭主库

    [root@mysql-server1 ~]# /etc/init.d/mysqld stop

    登录MyCAT

    [root@mysql-server1 ~]# mysql -utest -ptest -h127.0.0.1 -P8066 -DTESTDB

    插入数据

    复制代码
    mysql> insert into travelrecord(id,user_id,traveldate,fee,days)  values(10000001,@@hostname,20160103,100,10);
    ERROR 1184 (HY000): Connection refused
    mysql> select * from travelrecord;
    +---------+---------------+------------+------+------+
    | id      | user_id       | traveldate | fee  | days |
    +---------+---------------+------------+------+------+
    |       1 | mysql-server2 | 2016-01-01 |  100 |   10 |
    | 5000001 | mysql-server3 | 2016-01-02 |  100 |   10 |
    +---------+---------------+------------+------+------+
    2 rows in set (0.02 sec)
    复制代码

    可见无法插入数据,但不影响读取数据。

    至此,MyCAT实现MySQL的读写分离部署测试完毕。

    总结:

    1. 其实,刚开始配置的是readHost节点,配置如下:

    复制代码
     <dataHost name="localhost1" maxCon="1000" minCon="10" balance="1"
                    writeType="0" dbType="mysql" dbDriver="native" switchType="-1"  slaveThreshold="100">
                    <heartbeat>select user()</heartbeat>
                    <!-- can have multi write hosts -->
                    <writeHost host="hostM1" url="localhost:3306" user="root"
                            password="123456">
                            <!-- can have multi read hosts -->
                    <readHost host="hostS1" url="192.168.244.146:3306" user="root" password="123456" />
                    </writeHost>
    </dataHost>
    复制代码

    但这种方式有个问题,即master挂了以后,slave也不能提供服务,而这违反了MySQL主从集群的初衷。

    2. 如果开启了事务模式,即set autocommit=0,则事务内的读走的是master节点,而不是从节点。

     
    http://www.cnblogs.com/ivictor/p/5131480.html
     
     
  • 相关阅读:
    Python函数篇(7)-正则表达式
    Python函数篇(6)-常用模块及简单的案列
    Python函数篇(5)-装饰器及实例讲解
    Android 涂鸦最佳实践
    事件总线框架---Otto
    Android App补丁更新
    Android实现换肤功能(二)
    Android实现换肤功能(一)
    网络请求框架---Volley
    注解框架---AndroidAnnotations
  • 原文地址:https://www.cnblogs.com/softidea/p/5447566.html
Copyright © 2011-2022 走看看