MySQL/MariaDB数据库的主从复制
作者：尹正杰
版权声明：原创作品，谢绝转载！否则将追究法律责任。
一.MySQL复制概述
1>.传统扩展方式
垂直扩展(也叫向上扩展，Sacle Up): 在单节点通过提升服务器硬件性能以达到提升软件运行效率的问题，比如提升CPU,存储，内存，网络的性能。 在很多企业里面，MySQL数据库存放了一些重要的数据，当你的企业服务访问量越来越大,单个服务器性能以及很难达到要求了，一个单服务器的性能终究有限，通常在生产环境中一个数据库它的并发连接数能达到2000-3000已经很了不起了，在往上就撑不住了。 基于目前工艺条件所限，有可能单个服务器成本增大10倍，软件的性能才提升2倍。
水平扩展(也叫横向扩展，Scale Out):
增加多个相同配置的服务器对外提供服务，扩展起来相对简单，比如现在的分布式存储系统，集群等。
MySQL并发连接数有的时候并不是和服务器硬件有关的，而是MySQL数据库本身的性能就有极限。因此，当访问量达到一定规模的时候,单个数据库服务器性能压根扛不住。
目前基于工艺，成本考虑，可能更多的会考虑使用水平扩展。
2>.MySQL复制介绍
(1)MySQL复制允许将主实例（master）上的数据同步到一个或多个从实例（slave）上，默认情况下复制是异步进行的，从库也不需要一直连接主库来同步数据； (2)MySQL复制的数据粒度可以是主实例上所有的数据库，也可以是指定的一个或多个数据库，也可以是一个数据库里的指定的表； (3)MySQL复制有两种方法 传统方式 基于主库的bin-log将日志事件和事件位置复制到从库，从库再加以应用来达到主从同步的目的。 Gtid方式 global transaction identitifiers 是基于事物来复制数据，因此也就不依赖日志文件，同时又能更好的保证主从库数据一致性。
3>.MySQL复制带来的优势
扩展能力(读写分离，数据分布均匀以提升性能)：
通过复制可以将MySQL的读写分到一个或多个slave上。这要求所有的写操作和修改操作都必须在Master上完成，而读操作可以被分配到一个或多个salve上。将读写分离到不同服务器执行之后，MySQL的读写性能得到提升。
数据库备份：
由于从实例(slave)实时同步主实例(master)的数据，因此主实例(master)节点压力过大，此时可以将备份作业部署到从库。
需要注意的是，虽然主从复制能够起到一定程度上的备份，但并不能完全代替备份工作(一般MySQL集群都在同机房，建议做好异地备份,若该机房较长时间无法恢复通信,可以启用备份数据)。
数据分析和报表：
同样，一些数据分析和报表的实现可以在从实例执行，以减少对主库的性能影响。
容灾能力：
可以在物理距离较远的另一个数据建立slave,保证在主实例所在地区遭遇灾难时，在另一个数据中心能快速恢复。
高可用和故障切换:
通过MHA技术实现当master节点不正常工作了，其它slave节点可以自动选举出新的节点成为master,从而达到对master节点的备份效果。
MySQL升级测试:
在一个正常工作的MySQL集群中，若要对一个集群升级，可先下线一台SLAVE MySQL服务器，该节点下线并不影响对外提供服务，等服务器升级完毕后重新上线，等待该节点运行正常后，陆续下线其它节点并升级，整个过程并不影响MySQL集群的工作从而实现了平滑升级。
4>. MySQL复制有多种类型
异步复制 一个主库，一个或多个从库，数据异步同步到从库。 同步复制 在MySQL cluster中特有的复制方式。 半同步复制 在异步复制的基础上，确保任何一个主库上的事物在提交之前至少有一个从库已经收到该事物并日志记录下来。 延迟复制 在异步复制的基础上，人为设定主库和从库的数据同步延迟时间，即保证数据延迟至少是这个参数。
二.MySQL复制的常见架构
1>.一主一从
如下图所示:
用户向App Server服务器发送请求，App Server判断用户发来的请求是读还是写，如果是读就将SQL发送给slave节点响应，如果是写就将SQL发送给master节点响应,当master写完数据后将修改后的数据同步到slave节点已达到两个节点数据一致性的目的。
2>.一主多从
如下图所示：
其实架构和一主一从类似，只不过当master节点完成写操作后，会将数据同时同步到其它多个slave节点。
3>.MySQL垂直分区
主从复制基本上能够解决中小型企业的需求啦，但在很多场景下依然会遇到瓶颈问题，比如典型的双十一,淘宝很多用户下订单这些数据都会被记录到数据库,别说单台Mysql抗不住了，你直接使用单台Oracle数据库也未必能抗住啊。
因为主从复制架构中,只有master节点是负责写的，有的时候master节点写操作压力很大，性能也倒不倒要求了该怎么办?
典型的解决办法就是分库,分表,将数据拆开到不同节点分布式集群。而这个拆分方法就是我们所说的垂直分区和水平分片。
垂直拆分优点:
所谓垂直拆分指的是将同一个数据库的多张表分别存储到多台MySQL实例上存储;
本来同一个数据库的内容由之前1台服务器提供服务到现在的3台服务器提供服务，从而性能达到提升。
垂直拆分缺点:
若同一个数据库多张表在同一个实例上是可以进行一些内连接,外连接,关联查询等等，但由于我们将同一个数据库多张表分开存储就会导致我们无法执行各种join语句。
因此我们在拆分表的时候一定要考虑清除这些表是否有关系，是否会用到join查询，当然这个得和开发人员协商，因为表结构是它们设计的,哪些表有联系哪些表没有联系它们会更清楚。
4>.MySQL水平分片（Sharding）
如下图所示:
有"其它","用户","消息"这3张表，我们可以根据用户的id将表的内容进行分片处理，将奇数的用户数据从这3张表拆出来放在一个MySQL服务器上存储,把偶数的用户从这3张表中拆出来放在另外一台MySQL服务器上。
这也意味着每张大表都被拆分成2个小表各自放在不同的数据库上啦。
上面知识举个例子使用用户id来拆分表数据，当然咱们可以根据实际业务来进行从拆分，比如根据省份,用户积分等来拆分等。
如下图所示:
当user 103用户想要从网页查询他的数据，网页会通过前端传参给后端，后端通过ORM等技术转换成相应的SQL语句来查询数据库信息。
这个时候我们怎么知道user 103的数据存放在那个数据库实例上呢？
因此需要有一个分配管理器(Sharding Manager)来记录用户的数据在哪一台服务器上存放,相当于要增加一个类似调度器的东西。
虽然说数据库的性能得到提升，但是可以明显体会到系统的复杂度也提高了，需要加入更多的节点，此时任何一个节点出故障都会影响原来业务的使用,比如分片管理器不正常工作的话,尽管你数据库再多也没用，因为此时压根查不到数据在哪台节点上存放的，因此必须要想办法解决分配管理器的高可用性。这就得引入新的技术来实现,比如keepalive等技术。
三.MySQL主从复制原理
如下图所示:
1>.master节点收到写操作,通过用户权限验证后请求更新数据信息;
2>.如果是Innodb存储引擎会先写事务日志，写完事务日志以后再写数据文件，生成数据文件的同时会记录到二进制日志中，而Myisam压根就不支持事务,因此当接收到写请求时会直接修改数据从而在二进制日志中记录;
3>.master节点通过dump thread将生成的二进制日志读取出来并通过网络往从服务器发送;
4>.slave节点通过io thread将master的dump thread发送过来的数据接收并临时写入一个中继日志(Relay log)文件;
5>.slave节点通过SQL thread读取Relay log中的信息开始更新当前数据库数据信息。
1>.主从复制线程
主节点： dump Thread：
为每个Slave的I/O Thread启动一个dump线程，用于向其发送binary log events 从节点： I/O Thread：
向Master请求二进制日志事件，并保存于中继日志中 SQL Thread：
从中继日志中读取日志事件，在本地完成重放
2>.跟复制功能相关的文件
master.info：
用于保存slave连接至master时的相关信息，例如账号、密码、服务器地址等 relay-log.info：
保存在当前slave节点上已经复制的当前二进制日志和本地replay log日志的对应关系
3>.主从复制特点
异步复制
主从数据不一致比较常见
4>.复制架构
Master/Slave
Master/Master
环状复制
一主多从
从服务器还可以再有从服务器
一从多主:适用于多个不同数据库
5>.复制需要考虑二进制日志事件记录格式
基于语句的复制（statement based replication，5.0之前默认使用）： 基于主库将SQL语句写入到bin log中完成复制。 基于行数据的复制（row based replication,5.1之后推荐使用该模式）： 基于主库将每一行数据变化的信息作为时间写入到bin log中完成日志。默认就是基于行级别的复制，因为它相对语句复制逻辑更为严谨。 混合复制（mixed based replication）： 上述两者的结合。默认情况下优先使用基于语句的复制，只有当部分语句如果基于语句复制不完全的情况下才会自动切换为基于行数据的复制。
四.典型主从复制案例(未投入生产环境数据库，即2个数据库均为空)
1>.主从配置过程
博主推荐阅读: https://mariadb.com/kb/en/library/setting-up-replication/ https://dev.mysql.com/doc/refman/5.7/en/replication-configuration.html
2>.主节点配置
[root@node102.yinzhengjie.org.cn ~]# cat /etc/my.cnf [mysqld] binlog_format = row #推荐使用基于行数据的复制 log_bin = /data/logbin/mysql-bin #需要启用二进制日志,建议和数据文件分开放存放 server-id = 102 #为当前节点设置一个全局惟一的ID号,用于标识当前MySQL实例 log-basename = master #可选项，设置datadir中日志名称，确保不依赖主机名 character-set-server = utf8mb4 default_storage_engine = InnoDB datadir = /var/lib/mysql socket = /var/lib/mysql/mysql.sock [mysqld_safe] log-error = /var/log/mariadb/mariadb.log pid-file = /var/run/mariadb/mariadb.pid !includedir /etc/my.cnf.d [root@node102.yinzhengjie.org.cn ~]#
[root@node102.yinzhengjie.org.cn ~]# mysql -uroot -pyinzhengjie Welcome to the MariaDB monitor. Commands end with ; or g. Your MariaDB connection id is 6 Server version: 5.5.64-MariaDB MariaDB Server Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others. Type 'help;' or 'h' for help. Type 'c' to clear the current input statement. MariaDB [(none)]> MariaDB [(none)]> SELECT user,host,password FROM mysql.user; +------+-----------+-------------------------------------------+ | user | host | password | +------+-----------+-------------------------------------------+ | root | localhost | *BD0B1F48FDC55BD27555FC2F22FF29A68A25A1D7 | | root | 127.0.0.1 | *BD0B1F48FDC55BD27555FC2F22FF29A68A25A1D7 | | root | ::1 | *BD0B1F48FDC55BD27555FC2F22FF29A68A25A1D7 | +------+-----------+-------------------------------------------+ 3 rows in set (0.00 sec) MariaDB [(none)]> MariaDB [(none)]> GRANT REPLICATION SLAVE ON *.* TO 'copy'@'172.30.1.10%' IDENTIFIED BY 'yinzhengjie'; Query OK, 0 rows affected (0.00 sec) MariaDB [(none)]> MariaDB [(none)]> SELECT user,host,password FROM mysql.user; +------+--------------+-------------------------------------------+ | user | host | password | +------+--------------+-------------------------------------------+ | root | localhost | *BD0B1F48FDC55BD27555FC2F22FF29A68A25A1D7 | | root | 127.0.0.1 | *BD0B1F48FDC55BD27555FC2F22FF29A68A25A1D7 | | root | ::1 | *BD0B1F48FDC55BD27555FC2F22FF29A68A25A1D7 | | copy | 172.30.1.10% | *BD0B1F48FDC55BD27555FC2F22FF29A68A25A1D7 | +------+--------------+-------------------------------------------+ 4 rows in set (0.00 sec) MariaDB [(none)]>
MariaDB [(none)]> SHOW MASTER LOGS; #查看主节点的二进制日志当前所在位置 +-------------------+-----------+ | Log_name | File_size | +-------------------+-----------+ | master-bin.000001 | 245 | +-------------------+-----------+ 1 row in set (0.00 sec) MariaDB [(none)]>
3>.从节点配置
[root@node103.yinzhengjie.org.cn ~]# cat /etc/my.cnf [mysqld] server_id = 103 #为当前节点设置一个全局惟的ID号 read_only = ON #设置数据库只读 relay_log = relay-log #relay log的文件路径，默认值hostname-relay-bin relay_log_index = relay-log.index #默认值hostname-relay-bin.index datadir = /var/lib/mysql socket = /var/lib/mysql/mysql.sock [mysqld_safe] log-error=/var/log/mariadb/mariadb.log pid-file=/var/run/mariadb/mariadb.pid [root@node103.yinzhengjie.org.cn ~]#
[root@node103.yinzhengjie.org.cn ~]# mysql Welcome to the MariaDB monitor. Commands end with ; or g. Your MariaDB connection id is 2 Server version: 5.5.64-MariaDB MariaDB Server Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others. Type 'help;' or 'h' for help. Type 'c' to clear the current input statement. MariaDB [(none)]> MariaDB [(none)]> CHANGE MASTER TO -> MASTER_HOST='172.30.1.102', -> MASTER_USER='copy', -> MASTER_PASSWORD='yinzhengjie', -> MASTER_PORT=3306, -> MASTER_LOG_FILE='master-bin.000001', -> MASTER_LOG_POS=245, -> MASTER_CONNECT_RETRY=10; Query OK, 0 rows affected (0.00 sec) MariaDB [(none)]> MariaDB [(none)]>
MariaDB [(none)]> SHOW SLAVE STATUSG *************************** 1. row *************************** Slave_IO_State: Master_Host: 172.30.1.102 Master_User: copy Master_Port: 3306 Connect_Retry: 10 Master_Log_File: master-bin.000001 Read_Master_Log_Pos: 245 Relay_Log_File: relay-log.000001 Relay_Log_Pos: 4 Relay_Master_Log_File: master-bin.000001 Slave_IO_Running: No Slave_SQL_Running: No Replicate_Do_DB: Replicate_Ignore_DB: Replicate_Do_Table: Replicate_Ignore_Table: Replicate_Wild_Do_Table: Replicate_Wild_Ignore_Table: Last_Errno: 0 Last_Error: Skip_Counter: 0 Exec_Master_Log_Pos: 245 Relay_Log_Space: 245 Until_Condition: None Until_Log_File: Until_Log_Pos: 0 Master_SSL_Allowed: No Master_SSL_CA_File: Master_SSL_CA_Path: Master_SSL_Cert: Master_SSL_Cipher: Master_SSL_Key: Seconds_Behind_Master: NULL Master_SSL_Verify_Server_Cert: No Last_IO_Errno: 0 Last_IO_Error: Last_SQL_Errno: 0 Last_SQL_Error: Replicate_Ignore_Server_Ids: Master_Server_Id: 0 1 row in set (0.00 sec) MariaDB [(none)]> MariaDB [(none)]> SHOW PROCESSLIST; #未启动复制线程时，我们查看线程时看不到复制的IO线程的，等开启时迅速查看会有相应进程出现。 +----+------+-----------+------+---------+------+-------+------------------+----------+ | Id | User | Host | db | Command | Time | State | Info | Progress | +----+------+-----------+------+---------+------+-------+------------------+----------+ | 2 | root | localhost | NULL | Sleep | 69 | | NULL | 0.000 | | 3 | root | localhost | NULL | Query | 0 | NULL | SHOW PROCESSLIST | 0.000 | +----+------+-----------+------+---------+------+-------+------------------+----------+ 2 rows in set (0.00 sec) MariaDB [(none)]>
MariaDB [(none)]> SHOW SLAVE STATUSG *************************** 1. row *************************** Slave_IO_State: Master_Host: 172.30.1.102 Master_User: copy Master_Port: 3306 Connect_Retry: 10 Master_Log_File: master-bin.000001 Read_Master_Log_Pos: 245 Relay_Log_File: relay-log.000001 Relay_Log_Pos: 4 Relay_Master_Log_File: master-bin.000001 Slave_IO_Running: No Slave_SQL_Running: No Replicate_Do_DB: Replicate_Ignore_DB: Replicate_Do_Table: Replicate_Ignore_Table: Replicate_Wild_Do_Table: Replicate_Wild_Ignore_Table: Last_Errno: 0 Last_Error: Skip_Counter: 0 Exec_Master_Log_Pos: 245 Relay_Log_Space: 245 Until_Condition: None Until_Log_File: Until_Log_Pos: 0 Master_SSL_Allowed: No Master_SSL_CA_File: Master_SSL_CA_Path: Master_SSL_Cert: Master_SSL_Cipher: Master_SSL_Key: Seconds_Behind_Master: NULL Master_SSL_Verify_Server_Cert: No Last_IO_Errno: 0 Last_IO_Error: Last_SQL_Errno: 0 Last_SQL_Error: Replicate_Ignore_Server_Ids: Master_Server_Id: 0 1 row in set (0.00 sec) MariaDB [(none)]> MariaDB [(none)]> START SLAVE; #启动复制线程 Query OK, 0 rows affected (0.00 sec) MariaDB [(none)]> MariaDB [(none)]> MariaDB [(none)]> SHOW SLAVE STATUSG *************************** 1. row *************************** Slave_IO_State: Waiting for master to send event Master_Host: 172.30.1.102 Master_User: copy Master_Port: 3306 Connect_Retry: 10 Master_Log_File: master-bin.000001 Read_Master_Log_Pos: 245 Relay_Log_File: relay-log.000002 Relay_Log_Pos: 530 Relay_Master_Log_File: master-bin.000001 Slave_IO_Running: Yes #此处说明IO线程启动成功 Slave_SQL_Running: Yes #此处为yes说明SQL线程启动成功 Replicate_Do_DB: Replicate_Ignore_DB: Replicate_Do_Table: Replicate_Ignore_Table: Replicate_Wild_Do_Table: Replicate_Wild_Ignore_Table: Last_Errno: 0 Last_Error: Skip_Counter: 0 Exec_Master_Log_Pos: 245 Relay_Log_Space: 818 Until_Condition: None Until_Log_File: Until_Log_Pos: 0 Master_SSL_Allowed: No Master_SSL_CA_File: Master_SSL_CA_Path: Master_SSL_Cert: Master_SSL_Cipher: Master_SSL_Key: Seconds_Behind_Master: 0 #该参数表示延迟的秒数 Master_SSL_Verify_Server_Cert: No Last_IO_Errno: 0 Last_IO_Error: Last_SQL_Errno: 0 Last_SQL_Error: Replicate_Ignore_Server_Ids: Master_Server_Id: 102 1 row in set (0.00 sec) MariaDB [(none)]> MariaDB [(none)]> SHOW PROCESSLISTG *************************** 1. row *************************** Id: 2 User: root Host: localhost db: NULL Command: Sleep Time: 369 State: Info: NULL Progress: 0.000 *************************** 2. row *************************** Id: 3 User: root Host: localhost db: NULL Command: Query Time: 0 State: NULL Info: SHOW PROCESSLIST Progress: 0.000 *************************** 3. row *************************** Id: 4 User: system user Host: db: NULL Command: Connect Time: 540 State: Waiting for master to send event Info: NULL Progress: 0.000 *************************** 4. row *************************** Id: 5 User: system user Host: db: NULL Command: Connect Time: 540 State: Slave has read all relay log; waiting for the slave I/O thread to update it Info: NULL Progress: 0.000 4 rows in set (0.00 sec) MariaDB [(none)]>
4>.验证数据是否同步成功
[root@node103.yinzhengjie.org.cn ~]# mysql -e "SHOW DATABASES" #查看从库现有数据库 +--------------------+ | Database | +--------------------+ | information_schema | | mysql | | performance_schema | | test | +--------------------+ [root@node103.yinzhengjie.org.cn ~]#
[root@node102.yinzhengjie.org.cn ~]# mysql -uroot -pyinzhengjie Welcome to the MariaDB monitor. Commands end with ; or g. Your MariaDB connection id is 6 Server version: 5.5.64-MariaDB MariaDB Server Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others. Type 'help;' or 'h' for help. Type 'c' to clear the current input statement. MariaDB [(none)]> MariaDB [(none)]> CREATE DATABASE devops; Query OK, 1 row affected (0.00 sec) MariaDB [(none)]> MariaDB [(none)]> USE devops Database changed MariaDB [devops]> MariaDB [devops]> CREATE SCHEMA IF NOT EXISTS yinzhengjie2019 DEFAULT CHARACTER SET = utf8mb4; Query OK, 1 row affected (0.00 sec) MariaDB [devops]> MariaDB [devops]> CREATE TABLE students(id INT UNSIGNED AUTO_INCREMENT PRIMARY KEY,name VARCHAR(30) NOT NULL,sex ENUM('boy','girl') DEFAULT 'boy' ,age TINYINT UNSIGNED,mobile CHAR(11),address VARCHAR(50));Query OK, 0 rows affected (0.01 sec) MariaDB [devops]> INSERT INTO students (name,age,mobile,address) VALUES ('Jason Yin',18,10000,'beijing'),('Jay','40',10086,'Taiwan'); Query OK, 2 rows affected (0.00 sec) Records: 2 Duplicates: 0 Warnings: 0 MariaDB [devops]> INSERT INTO students SET name='yinzhengjie',age=27,address='shanxi'; Query OK, 1 row affected (0.01 sec) MariaDB [devops]> INSERT students (age,sex,name,mobile,address) VALUES (28,'girl','Gloria Tang Tsz-Kei',null,'Hong Kong'); Query OK, 1 row affected (0.00 sec) MariaDB [devops]> CREATE TABLE employee SELECT * FROM students; Query OK, 4 rows affected (0.00 sec) Records: 4 Duplicates: 0 Warnings: 0 MariaDB [devops]> MariaDB [devops]> SHOW TABLES; +------------------+ | Tables_in_devops | +------------------+ | employee | | students | +------------------+ 2 rows in set (0.00 sec) MariaDB [devops]> MariaDB [devops]> SELECT * FROM students; +----+---------------------+------+------+--------+-----------+ | id | name | sex | age | mobile | address | +----+---------------------+------+------+--------+-----------+ | 1 | Jason Yin | boy | 18 | 10000 | beijing | | 2 | Jay | boy | 40 | 10086 | Taiwan | | 3 | yinzhengjie | boy | 27 | NULL | shanxi | | 4 | Gloria Tang Tsz-Kei | girl | 28 | NULL | Hong Kong | +----+---------------------+------+------+--------+-----------+ 4 rows in set (0.00 sec) MariaDB [devops]> MariaDB [devops]> SELECT * FROM employee; +----+---------------------+------+------+--------+-----------+ | id | name | sex | age | mobile | address | +----+---------------------+------+------+--------+-----------+ | 1 | Jason Yin | boy | 18 | 10000 | beijing | | 2 | Jay | boy | 40 | 10086 | Taiwan | | 3 | yinzhengjie | boy | 27 | NULL | shanxi | | 4 | Gloria Tang Tsz-Kei | girl | 28 | NULL | Hong Kong | +----+---------------------+------+------+--------+-----------+ 4 rows in set (0.00 sec) MariaDB [devops]> MariaDB [devops]> SHOW DATABASES; +--------------------+ | Database | +--------------------+ | information_schema | | devops | | mysql | | performance_schema | | test | | yinzhengjie2019 | +--------------------+ 6 rows in set (0.00 sec) MariaDB [devops]> MariaDB [devops]> QUIT Bye [root@node102.yinzhengjie.org.cn ~]# [root@node102.yinzhengjie.org.cn ~]#
[root@node103.yinzhengjie.org.cn ~]# mysql -e "SHOW DATABASES" +--------------------+ | Database | +--------------------+ | information_schema | | devops | | mysql | | performance_schema | | test | | yinzhengjie2019 | +--------------------+ [root@node103.yinzhengjie.org.cn ~]# [root@node103.yinzhengjie.org.cn ~]# mysql -e "USE devops;SHOW TABLES" +------------------+ | Tables_in_devops | +------------------+ | employee | | students | +------------------+ [root@node103.yinzhengjie.org.cn ~]# [root@node103.yinzhengjie.org.cn ~]# mysql -e "USE devops;SELECT * FROM employee" +----+---------------------+------+------+--------+-----------+ | id | name | sex | age | mobile | address | +----+---------------------+------+------+--------+-----------+ | 1 | Jason Yin | boy | 18 | 10000 | beijing | | 2 | Jay | boy | 40 | 10086 | Taiwan | | 3 | yinzhengjie | boy | 27 | NULL | shanxi | | 4 | Gloria Tang Tsz-Kei | girl | 28 | NULL | Hong Kong | +----+---------------------+------+------+--------+-----------+ [root@node103.yinzhengjie.org.cn ~]# [root@node103.yinzhengjie.org.cn ~]# mysql -e "USE devops;SELECT * FROM students" +----+---------------------+------+------+--------+-----------+ | id | name | sex | age | mobile | address | +----+---------------------+------+------+--------+-----------+ | 1 | Jason Yin | boy | 18 | 10000 | beijing | | 2 | Jay | boy | 40 | 10086 | Taiwan | | 3 | yinzhengjie | boy | 27 | NULL | shanxi | | 4 | Gloria Tang Tsz-Kei | girl | 28 | NULL | Hong Kong | +----+---------------------+------+------+--------+-----------+ [root@node103.yinzhengjie.org.cn ~]#
[root@node102.yinzhengjie.org.cn ~]# mysql -uroot -pyinzhengjie Welcome to the MariaDB monitor. Commands end with ; or g. Your MariaDB connection id is 7 Server version: 5.5.64-MariaDB MariaDB Server Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others. Type 'help;' or 'h' for help. Type 'c' to clear the current input statement. MariaDB [(none)]> use yinzhengjie2019 Database changed MariaDB [yinzhengjie2019]> MariaDB [yinzhengjie2019]> CREATE TABLE testlog (id INT auto_increment PRIMARY KEY,name CHAR(50),age INT DEFAULT 20); Query OK, 0 rows affected (0.01 sec) MariaDB [yinzhengjie2019]> MariaDB [yinzhengjie2019]> DELIMITER $$ MariaDB [yinzhengjie2019]> MariaDB [yinzhengjie2019]> CREATE PROCEDURE pro_testlog() -> BEGIN -> DECLARE i INT; -> SET i = 1; -> WHILE i < 100000 -> DO -> INSERT INTO testlog(name,age) VALUES (CONCAT('yinzhengjie',i),i); -> SET i = i +1; -> END WHILE; -> END$$ Query OK, 0 rows affected (0.00 sec) MariaDB [yinzhengjie2019]> MariaDB [yinzhengjie2019]> DELIMITER ; MariaDB [yinzhengjie2019]> MariaDB [yinzhengjie2019]> SHOW PROCEDURE STATUSG *************************** 1. row *************************** Db: yinzhengjie2019 Name: pro_testlog Type: PROCEDURE Definer: root@localhost Modified: 2019-11-08 12:04:06 Created: 2019-11-08 12:04:06 Security_type: DEFINER Comment: character_set_client: utf8 collation_connection: utf8_general_ci Database Collation: utf8mb4_general_ci 1 row in set (0.00 sec) MariaDB [yinzhengjie2019]> MariaDB [yinzhengjie2019]>
[root@node103.yinzhengjie.org.cn ~]# mysql -e "USE yinzhengjie2019;SHOW TABLES" +---------------------------+ | Tables_in_yinzhengjie2019 | +---------------------------+ | testlog | +---------------------------+ [root@node103.yinzhengjie.org.cn ~]# [root@node103.yinzhengjie.org.cn ~]# mysql -e "SHOW PROCEDURE STATUSG" *************************** 1. row *************************** Db: yinzhengjie2019 Name: pro_testlog Type: PROCEDURE Definer: root@localhost Modified: 2019-11-08 12:04:06 Created: 2019-11-08 12:04:06 Security_type: DEFINER Comment: character_set_client: utf8 collation_connection: utf8_general_ci Database Collation: utf8mb4_general_ci [root@node103.yinzhengjie.org.cn ~]#
[root@node102.yinzhengjie.org.cn ~]# mysql -uroot -pyinzhengjie yinzhengjie2019 Reading table information for completion of table and column names You can turn off this feature to get a quicker startup with -A Welcome to the MariaDB monitor. Commands end with ; or g. Your MariaDB connection id is 8 Server version: 5.5.64-MariaDB MariaDB Server Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others. Type 'help;' or 'h' for help. Type 'c' to clear the current input statement. MariaDB [yinzhengjie2019]> MariaDB [yinzhengjie2019]> SELECT COUNT(*) FROM testlog; +----------+ | COUNT(*) | +----------+ | 0 | +----------+ 1 row in set (0.00 sec) MariaDB [yinzhengjie2019]> MariaDB [yinzhengjie2019]> CALL pro_testlog; Query OK, 1 row affected (59.41 sec) MariaDB [yinzhengjie2019]> CALL pro_testlog; Query OK, 1 row affected (58.75 sec) MariaDB [yinzhengjie2019]> MariaDB [yinzhengjie2019]> CALL pro_testlog; Query OK, 1 row affected (59.01 sec) MariaDB [yinzhengjie2019]> MariaDB [yinzhengjie2019]> CALL pro_testlog; Query OK, 1 row affected (58.75 sec) MariaDB [yinzhengjie2019]> MariaDB [yinzhengjie2019]> SELECT COUNT(*) FROM yinzhengjie2019.testlog; +----------+ | COUNT(*) | +----------+ | 399996 | +----------+ 1 row in set (0.09 sec) MariaDB [yinzhengjie2019]> MariaDB [yinzhengjie2019]> SHOW PROCESSLISTG *************************** 1. row *************************** Id: 4 User: copy Host: node103.yinzhengjie.org.cn:49672 db: NULL Command: Binlog Dump Time: 2167 State: Master has sent all binlog to slave; waiting for binlog to be updated Info: NULL Progress: 0.000 *************************** 2. row *************************** Id: 8 User: root Host: localhost db: yinzhengjie2019 Command: Query Time: 0 State: NULL Info: SHOW PROCESSLIST Progress: 0.000 2 rows in set (0.00 sec) MariaDB [yinzhengjie2019]> MariaDB [yinzhengjie2019]>
MariaDB [(none)]> SHOW SLAVE STATUSG *************************** 1. row *************************** Slave_IO_State: Waiting for master to send event Master_Host: 172.30.1.102 Master_User: copy Master_Port: 3306 Connect_Retry: 10 Master_Log_File: master-bin.000001 Read_Master_Log_Pos: 73098208 Relay_Log_File: relay-log.000002 Relay_Log_Pos: 73098271 Relay_Master_Log_File: master-bin.000001 Slave_IO_Running: Yes Slave_SQL_Running: Yes Replicate_Do_DB: Replicate_Ignore_DB: Replicate_Do_Table: Replicate_Ignore_Table: Replicate_Wild_Do_Table: Replicate_Wild_Ignore_Table: Last_Errno: 0 Last_Error: Skip_Counter: 0 Exec_Master_Log_Pos: 73097986 Relay_Log_Space: 73098781 Until_Condition: None Until_Log_File: Until_Log_Pos: 0 Master_SSL_Allowed: No Master_SSL_CA_File: Master_SSL_CA_Path: Master_SSL_Cert: Master_SSL_Cipher: Master_SSL_Key: Seconds_Behind_Master: 1 #发现此时从库的数据和主库不一致啦,数据同步的速度较慢,过一段时间我们发现这个数字又为0啦！说明数据同步成功啦！ Master_SSL_Verify_Server_Cert: No Last_IO_Errno: 0 Last_IO_Error: Last_SQL_Errno: 0 Last_SQL_Error: Replicate_Ignore_Server_Ids: Master_Server_Id: 102 1 row in set (0.00 sec) MariaDB [(none)]> SELECT COUNT(*) FROM yinzhengjie2019.testlog; #不难发现，从库的消息条数和主库不一致，目前还处于正在同步的状态 +----------+ | COUNT(*) | +----------+ | 379301 | +----------+ 1 row in set (0.11 sec) MariaDB [(none)]> MariaDB [(none)]> SHOW SLAVE STATUSG *************************** 1. row *************************** Slave_IO_State: Waiting for master to send event Master_Host: 172.30.1.102 Master_User: copy Master_Port: 3306 Connect_Retry: 10 Master_Log_File: master-bin.000001 Read_Master_Log_Pos: 88757200 Relay_Log_File: relay-log.000002 Relay_Log_Pos: 88757485 Relay_Master_Log_File: master-bin.000001 Slave_IO_Running: Yes Slave_SQL_Running: Yes Replicate_Do_DB: Replicate_Ignore_DB: Replicate_Do_Table: Replicate_Ignore_Table: Replicate_Wild_Do_Table: Replicate_Wild_Ignore_Table: Last_Errno: 0 Last_Error: Skip_Counter: 0 Exec_Master_Log_Pos: 88757200 Relay_Log_Space: 88757773 Until_Condition: None Until_Log_File: Until_Log_Pos: 0 Master_SSL_Allowed: No Master_SSL_CA_File: Master_SSL_CA_Path: Master_SSL_Cert: Master_SSL_Cipher: Master_SSL_Key: Seconds_Behind_Master: 0 #此时从库和主库的数据是一致的啦！ Master_SSL_Verify_Server_Cert: No Last_IO_Errno: 0 Last_IO_Error: Last_SQL_Errno: 0 Last_SQL_Error: Replicate_Ignore_Server_Ids: Master_Server_Id: 102 1 row in set (0.00 sec) MariaDB [(none)]> MariaDB [(none)]> SELECT COUNT(*) FROM yinzhengjie2019.testlog; #消息行数和主库一致 +----------+ | COUNT(*) | +----------+ | 399996 | +----------+ 1 row in set (0.10 sec) MariaDB [(none)]>
五.模拟生产环境中主从复制(主节点已经运行了一段时间且有大量数据时，如何配置并启动slave节点)
1>.主节点备份现有数据(我们依旧上面的"node102.yinzhengjie.org.cn"节点作为master)
[root@node102.yinzhengjie.org.cn ~]# cat /etc/hosts 127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 ::1 localhost localhost.localdomain localhost6 localhost6.localdomain6 172.30.1.101 node101.yinzhengjie.org.cn 172.30.1.102 node102.yinzhengjie.org.cn 172.30.1.103 node103.yinzhengjie.org.cn 172.30.1.104 node104.yinzhengjie.org.cn 172.30.1.105 node105.yinzhengjie.org.cn 172.30.1.106 node106.yinzhengjie.org.cn 172.30.1.107 node107.yinzhengjie.org.cn 172.30.1.108 node108.yinzhengjie.org.cn [root@node102.yinzhengjie.org.cn ~]#
[root@node102.yinzhengjie.org.cn ~]# cat /etc/my.cnf #主节点配置并不需要修改 [mysqld] binlog_format = row log_bin = /data/logbin/mysql-bin server-id = 102 log-basename = master character-set-server = utf8mb4 default_storage_engine = InnoDB datadir = /var/lib/mysql socket = /var/lib/mysql/mysql.sock [mysqld_safe] log-error = /var/log/mariadb/mariadb.log pid-file = /var/run/mariadb/mariadb.pid !includedir /etc/my.cnf.d [root@node102.yinzhengjie.org.cn ~]#
[root@node102.yinzhengjie.org.cn ~]# mysqldump -uroot -pyinzhengjie -A -F --single-transaction --master-data=1 > /root/all_bak.sql [root@node102.yinzhengjie.org.cn ~]# [root@node102.yinzhengjie.org.cn ~]# ll total 13596 -rw-r--r-- 1 root root 13918698 Nov 8 12:41 all_bak.sql [root@node102.yinzhengjie.org.cn ~]# [root@node102.yinzhengjie.org.cn ~]# scp all_bak.sql node104.yinzhengjie.org.cn:~ The authenticity of host 'node104.yinzhengjie.org.cn (172.30.1.104)' can't be established. ECDSA key fingerprint is SHA256:F3IVf82keybIystuO6PYRfwr0o5dTftrmAHJWzqO4IA. ECDSA key fingerprint is MD5:02:5d:d8:0a:4a:b4:70:0f:61:be:2c:97:56:db:24:e7. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added 'node104.yinzhengjie.org.cn,172.30.1.104' (ECDSA) to the list of known hosts. root@node104.yinzhengjie.org.cn's password: all_bak.sql 100% 13MB 74.2MB/s 00:00 [root@node102.yinzhengjie.org.cn ~]#
2>.从节点配置(此时我们引入新节点"node104.yinzhengjie.org.cn")
[root@node104.yinzhengjie.org.cn ~]# ll -h total 14M -rw-r--r-- 1 root root 14M Nov 8 12:43 all_bak.sql [root@node104.yinzhengjie.org.cn ~]# [root@node104.yinzhengjie.org.cn ~]# head -25 all_bak.sql | tail /*!40111 SET @OLD_SQL_NOTES=@@SQL_NOTES, SQL_NOTES=0 */; -- -- Position to start replication or point-in-time recovery from -- CHANGE MASTER TO MASTER_LOG_FILE='master-bin.000002', MASTER_LOG_POS=245; -- -- Current Database: `devops` [root@node104.yinzhengjie.org.cn ~]# [root@node104.yinzhengjie.org.cn ~]# vim all_bak.sql #需要传过来的备份文件稍作修改，主要是配置使用有复制权限的用户账号连接至主服务器 [root@node104.yinzhengjie.org.cn ~]# [root@node104.yinzhengjie.org.cn ~]# head -30 all_bak.sql | tail #查看修改后的内容 CHANGE MASTER TO MASTER_HOST='172.30.1.102', MASTER_USER='copy', MASTER_PASSWORD='yinzhengjie', MASTER_PORT=3306, MASTER_CONNECT_RETRY=10, MASTER_LOG_FILE='master-bin.000002', MASTER_LOG_POS=245; [root@node104.yinzhengjie.org.cn ~]# [root@node104.yinzhengjie.org.cn ~]#
[root@node104.yinzhengjie.org.cn ~]# ll /var/lib/mysql/ total 0 [root@node104.yinzhengjie.org.cn ~]# [root@node104.yinzhengjie.org.cn ~]# systemctl start mariadb [root@node104.yinzhengjie.org.cn ~]# [root@node104.yinzhengjie.org.cn ~]# ll /var/lib/mysql/ total 37852 -rw-rw---- 1 mysql mysql 16384 Nov 8 12:55 aria_log.00000001 -rw-rw---- 1 mysql mysql 52 Nov 8 12:55 aria_log_control -rw-rw---- 1 mysql mysql 18874368 Nov 8 12:55 ibdata1 -rw-rw---- 1 mysql mysql 5242880 Nov 8 12:55 ib_logfile0 -rw-rw---- 1 mysql mysql 5242880 Nov 8 12:55 ib_logfile1 drwx------ 2 mysql mysql 4096 Nov 8 12:55 mysql srwxrwxrwx 1 mysql mysql 0 Nov 8 12:55 mysql.sock drwx------ 2 mysql mysql 4096 Nov 8 12:55 performance_schema drwx------ 2 mysql mysql 6 Nov 8 12:55 test [root@node104.yinzhengjie.org.cn ~]# [root@node104.yinzhengjie.org.cn ~]# ll total 13596 -rw-r--r-- 1 root root 13918853 Nov 8 12:50 all_bak.sql [root@node104.yinzhengjie.org.cn ~]# [root@node104.yinzhengjie.org.cn ~]# mysql < all_bak.sql #启动MySQL服务后并导入咱们修改的备份文件 [root@node104.yinzhengjie.org.cn ~]# [root@node104.yinzhengjie.org.cn ~]# ll /var/lib/mysql/ total 70636 -rw-rw---- 1 mysql mysql 16384 Nov 8 12:55 aria_log.00000001 -rw-rw---- 1 mysql mysql 52 Nov 8 12:55 aria_log_control drwx------ 2 mysql mysql 60 Nov 8 12:55 devops -rw-rw---- 1 mysql mysql 52428800 Nov 8 12:55 ibdata1 -rw-rw---- 1 mysql mysql 5242880 Nov 8 12:55 ib_logfile0 -rw-rw---- 1 mysql mysql 5242880 Nov 8 12:55 ib_logfile1 -rw-rw---- 1 mysql mysql 84 Nov 8 12:55 master.info drwx------ 2 mysql mysql 4096 Nov 8 12:55 mysql srwxrwxrwx 1 mysql mysql 0 Nov 8 12:55 mysql.sock drwx------ 2 mysql mysql 4096 Nov 8 12:55 performance_schema -rw-rw---- 1 mysql mysql 245 Nov 8 12:55 relay-log.000001 -rw-rw---- 1 mysql mysql 19 Nov 8 12:55 relay-log.index -rw-rw---- 1 mysql mysql 43 Nov 8 12:55 relay-log.info drwx------ 2 mysql mysql 6 Nov 8 12:55 test drwx------ 2 mysql mysql 39 Nov 8 12:55 yinzhengjie2019 [root@node104.yinzhengjie.org.cn ~]# [root@node104.yinzhengjie.org.cn ~]# cat /var/lib/mysql/master.info 18 master-bin.000002 245 172.30.1.102 copy yinzhengjie 3306 10 0 0 1800.000 0 [root@node104.yinzhengjie.org.cn ~]# [root@node104.yinzhengjie.org.cn ~]# [root@node104.yinzhengjie.org.cn ~]# cat /var/lib/mysql/relay-log.info ./relay-log.000001 4 master-bin.000002 245 [root@node104.yinzhengjie.org.cn ~]# [root@node104.yinzhengjie.org.cn ~]#
[root@node104.yinzhengjie.org.cn ~]# mysql Welcome to the MariaDB monitor. Commands end with ; or g. Your MariaDB connection id is 3 Server version: 5.5.64-MariaDB MariaDB Server Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others. Type 'help;' or 'h' for help. Type 'c' to clear the current input statement. MariaDB [(none)]> MariaDB [(none)]> MariaDB [(none)]> SHOW SLAVE STATUSG *************************** 1. row *************************** Slave_IO_State: Master_Host: 172.30.1.102 Master_User: copy Master_Port: 3306 Connect_Retry: 10 Master_Log_File: master-bin.000002 Read_Master_Log_Pos: 245 Relay_Log_File: relay-log.000001 Relay_Log_Pos: 4 Relay_Master_Log_File: master-bin.000002 Slave_IO_Running: No Slave_SQL_Running: No Replicate_Do_DB: Replicate_Ignore_DB: Replicate_Do_Table: Replicate_Ignore_Table: Replicate_Wild_Do_Table: Replicate_Wild_Ignore_Table: Last_Errno: 0 Last_Error: Skip_Counter: 0 Exec_Master_Log_Pos: 245 Relay_Log_Space: 245 Until_Condition: None Until_Log_File: Until_Log_Pos: 0 Master_SSL_Allowed: No Master_SSL_CA_File: Master_SSL_CA_Path: Master_SSL_Cert: Master_SSL_Cipher: Master_SSL_Key: Seconds_Behind_Master: NULL Master_SSL_Verify_Server_Cert: No Last_IO_Errno: 0 Last_IO_Error: Last_SQL_Errno: 0 Last_SQL_Error: Replicate_Ignore_Server_Ids: Master_Server_Id: 0 1 row in set (0.00 sec) MariaDB [(none)]> MariaDB [(none)]> MariaDB [(none)]> START SLAVE; #启动复制线程 Query OK, 0 rows affected (0.00 sec) MariaDB [(none)]> MariaDB [(none)]> SHOW SLAVE STATUSG *************************** 1. row *************************** Slave_IO_State: Waiting for master to send event Master_Host: 172.30.1.102 Master_User: copy Master_Port: 3306 Connect_Retry: 10 Master_Log_File: master-bin.000002 Read_Master_Log_Pos: 245 Relay_Log_File: relay-log.000002 Relay_Log_Pos: 530 Relay_Master_Log_File: master-bin.000002 Slave_IO_Running: Yes Slave_SQL_Running: Yes Replicate_Do_DB: Replicate_Ignore_DB: Replicate_Do_Table: Replicate_Ignore_Table: Replicate_Wild_Do_Table: Replicate_Wild_Ignore_Table: Last_Errno: 0 Last_Error: Skip_Counter: 0 Exec_Master_Log_Pos: 245 Relay_Log_Space: 818 Until_Condition: None Until_Log_File: Until_Log_Pos: 0 Master_SSL_Allowed: No Master_SSL_CA_File: Master_SSL_CA_Path: Master_SSL_Cert: Master_SSL_Cipher: Master_SSL_Key: Seconds_Behind_Master: 0 Master_SSL_Verify_Server_Cert: No Last_IO_Errno: 0 Last_IO_Error: Last_SQL_Errno: 0 Last_SQL_Error: Replicate_Ignore_Server_Ids: Master_Server_Id: 102 1 row in set (0.00 sec) MariaDB [(none)]>
3>.验证数据是否同步成功
[root@node102.yinzhengjie.org.cn ~]# mysql -uroot -pyinzhengjie yinzhengjie2019 Reading table information for completion of table and column names You can turn off this feature to get a quicker startup with -A Welcome to the MariaDB monitor. Commands end with ; or g. Your MariaDB connection id is 12 Server version: 5.5.64-MariaDB MariaDB Server Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others. Type 'help;' or 'h' for help. Type 'c' to clear the current input statement. MariaDB [yinzhengjie2019]> MariaDB [yinzhengjie2019]> MariaDB [yinzhengjie2019]> SELECT COUNT(*) FROM yinzhengjie2019.testlog; +----------+ | COUNT(*) | +----------+ | 399996 | +----------+ 1 row in set (0.17 sec) MariaDB [yinzhengjie2019]> MariaDB [yinzhengjie2019]> CALL pro_testlog; Query OK, 1 row affected (1 min 24.17 sec) MariaDB [yinzhengjie2019]> MariaDB [yinzhengjie2019]> CALL pro_testlog; Query OK, 1 row affected (1 min 21.85 sec) MariaDB [yinzhengjie2019]> MariaDB [yinzhengjie2019]> SELECT COUNT(*) FROM yinzhengjie2019.testlog; +----------+ | COUNT(*) | +----------+ | 599994 | +----------+ 1 row in set (0.14 sec) MariaDB [yinzhengjie2019]> MariaDB [yinzhengjie2019]>
[root@node104.yinzhengjie.org.cn ~]# mysql Welcome to the MariaDB monitor. Commands end with ; or g. Your MariaDB connection id is 7 Server version: 5.5.64-MariaDB MariaDB Server Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others. Type 'help;' or 'h' for help. Type 'c' to clear the current input statement. MariaDB [(none)]> MariaDB [(none)]> SELECT COUNT(*) FROM yinzhengjie2019.testlog; +----------+ | COUNT(*) | +----------+ | 599994 | +----------+ 1 row in set (0.14 sec) MariaDB [(none)]> MariaDB [(none)]>