作业第十四周

 #### 1 MariaDB主从复制原理

从库生成两个线程，一个I/O线程，一个SQL线程；

 

i/o线程去请求主库 的binlog，并将得到的binlog日志写到relay log（中继日志） 文件中；

主库会生成一个 log dump 线程，用来给从库 i/o线程传binlog；

 

SQL 线程，会读取relay log文件中的日志，并解析成具体操作，来实现主从的操作一致，而最终数据一致；

 

 

#### 2 MariaDB一主一从架构构建
主：
yum -y install mariadb-server
[root@centos8-1 ~]$vim /etc/my.cnf.d/mariadb-server.cnf
##在[mysqld] 中添加如下字段
log-bin ## 开启二进制日志
server-id=8 ##指定唯一服务器ID
:保存退出
[root@centos8-1 ~]$systemctl start mariadb
[root@centos8-1 ~]$mysql
MariaDB [(none)]> show master logs;
+--------------------+-----------+
| Log_name | File_size |
+--------------------+-----------+
| mariadb-bin.000001 | 28434 |
| mariadb-bin.000002 | 344 |
+--------------------+-----------+
2 rows in set (0.000 sec)
MariaDB [(none)]> grant replication slave on *.* to repluser@'10.0.0.%' identified by 'tianze';
MariaDB [(none)]> flush privileges;
从：
yum -y install mariadb-server
[root@centos8-2 ~]$vim /etc/my.cnf.d/mariadb-server.cnf
##在[mysqld] 中添加如下字段
read-only ## 设置从服务器只读
server-id=18 ##指定唯一服务器ID
:保存退出
[root@centos8-2 ~]$systemctl start mariadb
[root@centos8-2 ~]$mysql
MariaDB [(none)]> change master to master_host='10.0.0.8',master_user='repluser',master_password='tianze',master_port=3306,master_log_file='mariadb-bin.000002',master_log_pos=344;
MariaDB [(none)]> start slave;
Query OK, 0 rows affected (0.002 sec)

MariaDB [(none)]> show slave statusG
*************************** 1. row ***************************
Slave_IO_State: Waiting for master to send event
Master_Host: 10.0.0.8
Master_User: repluser
Master_Port: 3306
Connect_Retry: 60
Master_Log_File: mariadb-bin.000002
Read_Master_Log_Pos: 672
Relay_Log_File: mariadb-relay-bin.000002
Relay_Log_Pos: 885
Relay_Master_Log_File: mariadb-bin.000002
Slave_IO_Running: Yes
Slave_SQL_Running: Yes
Replicate_Do_DB:
Replicate_Ignore_DB:
Replicate_Do_Table:
Replicate_Ignore_Table:
Replicate_Wild_Do_Table:
Replicate_Wild_Ignore_Table:
Last_Errno: 0
Last_Error:
Skip_Counter: 0
Exec_Master_Log_Pos: 672
Relay_Log_Space: 1196
Until_Condition: None
Until_Log_File:
Until_Log_Pos: 0
Master_SSL_Allowed: No
Master_SSL_CA_File:
Master_SSL_CA_Path:
Master_SSL_Cert:
Master_SSL_Cipher:
Master_SSL_Key:
Seconds_Behind_Master: 0
Master_SSL_Verify_Server_Cert: No
Last_IO_Errno: 0
Last_IO_Error:
Last_SQL_Errno: 0
Last_SQL_Error:
Replicate_Ignore_Server_Ids:
Master_Server_Id: 8
Master_SSL_Crl:
Master_SSL_Crlpath:
Using_Gtid: No
Gtid_IO_Pos:
Replicate_Do_Domain_Ids:
Replicate_Ignore_Domain_Ids:
Parallel_Mode: conservative
SQL_Delay: 0
SQL_Remaining_Delay: NULL
Slave_SQL_Running_State: Slave has read all relay log; waiting for the slave I/O thread to update it
Slave_DDL_Groups: 2
Slave_Non_Transactional_Groups: 0
Slave_Transactional_Groups: 0
1 row in set (0.000 sec)

 

准备两台纯新的CentOS7.6服务器，其中主机称为node1的服务器地址为192.168.130.132，主机称为node2的服务器地址为192.168.130.133

#### 3 MariaDB级联复制

 

三台Centos8主机实现级别复制 10.0.0.8:Master 10.0.0.18:Slave1 10.0.0.28:Slave2

全部安装Mariadb：yum -y install mariadb-server

在Master实现
[root@master ~]# vim /etc/my.cnf.d/mariadb-server.cnf
##[mysqld]中 添加
server-id=1
log-bin

[root@master ~]# systemctl start mariadb
[root@master ~]# mysql
MariaDB [(none)]> show master logs;
+--------------------+-----------+
| Log_name | File_size |
+--------------------+-----------+
| mariadb-bin.000001 | 28198 |
| mariadb-bin.000002 | 344 |
+--------------------+-----------+
2 rows in set (0.000 sec)

MariaDB [(none)]> grant replication slave on*.* to repluser@'10.0.0.%' identified by 'tianze';
Query OK, 0 rows affected (0.000 sec)

MariaDB [(none)]> flush privileges;
Query OK, 0 rows affected (0.000 sec)

[root@master ~]# mysqldump -A -F --single-transaction --master-data=1 > /data/all_`date +%F`.sql

[root@master ~]# scp /data/all_2020-09-03.sql 10.0.0.18:/data
[root@master ~]# scp /data/all_2020-09-03.sql 10.0.0.28:/data

在中间级联Slave1实现
[root@slave1 ~]# vim /etc/my.cnf.d/mariadb-server.cnf
##[mysqld]中 添加
server-id=2
log-bin
read-only
log_slave_updates ##级联复制中间节点的必选项

[root@slave1 ~]# systemctl start mariadb

##还原数据库
[root@slave1 ~]# vim /data/all_2020-09-03.sql
CHANGE MASTER TO MASTER_HOST='10.0.0.8',MASTER_USER='repluser',MASTER_PASSWORD='tianze',MASTER_PORT=3306,MASTER_LOG_FILE='mariadb-bin.000003', MASTER_LOG_POS=389;
[root@slave1 ~]# mysql
MariaDB [(none)]> set sql_log_bin=0;
Query OK, 0 rows affected (0.000 sec)
MariaDB [mysql]> source /data/all_2020-09-03.sql
MariaDB [mysql]> show master logs; ## 记录二进制日志给第三个节点同步使用
+--------------------+-----------+
| Log_name | File_size |
+--------------------+-----------+
| mariadb-bin.000001 | 28198 |
| mariadb-bin.000002 | 663 |
+--------------------+-----------+
2 rows in set (0.000 sec)

MariaDB [mysql]> set sql_log_bin=1; ##恢复开启二进制日志
MariaDB [mysql]> start slave; ##开启同步
MariaDB [mysql]> show slave statusG ##查看同步状态

在第三个节点配置
[root@slave2 ~]# vim /etc/my.cnf.d/mariadb-server.cnf
##[mysqld]中 添加
server-id=3
read-only

[root@slave2 ~]# systemctl start mariadb.service
[root@slave2 ~]# vim /data/all_2020-09-03.sql

CHANGE MASTER TO MASTER_HOST='10.0.0.18',MASTER_USER='repluser',MASTER_PASSWORD='tianze',MASTER_PORT=3306,MASTER_LOG_FILE='mariadb-bin.000002', MASTER_LOG_POS=663;

[root@slave2 ~]# mysql
MariaDB [(none)]> source /data/all_2020-09-03.sql
MariaDB [mysql]> start slave; ##开启同步
MariaDB [mysql]> show slave statusG ##查看同步状态

##测试同步 在第一个master创建数据库
MariaDB [(none)]> create database test;
Query OK, 1 row affected (0.000 sec)

在第三个节点查看数据库，发现已经同步，配置完成
MariaDB [mysql]> show databases;
+--------------------+
| Database |
+--------------------+
| test |
| information_schema |
| mysql |
| performance_schema |

 

 

 

#### 4 MariaDB半同步复制
主：
yum -y install mariadb-server
[root@centos8-1 ~]$vim /etc/my.cnf.d/mariadb-server.cnf
##在[mysqld] 中添加如下字段
log-bin ## 开启二进制日志
server-id=8 ##指定唯一服务器ID
plugin-load-add=semisync_master
rpl_semi_sync_master_enabled=ON
rpl_semi_sync_master_timeout=3000
:保存退出
[root@centos8-1 ~]$systemctl restart mariadb
[root@centos8-1 ~]$mysql
MariaDB [test]> show global variables like '%semi%';
ERROR 2006 (HY000): MySQL server has gone away
No connection. Trying to reconnect...
Connection id: 10
Current database: test

+---------------------------------------+--------------+
| Variable_name | Value |
+---------------------------------------+--------------+
| rpl_semi_sync_master_enabled | ON |
| rpl_semi_sync_master_timeout | 3000 |
| rpl_semi_sync_master_trace_level | 32 |
| rpl_semi_sync_master_wait_no_slave | ON |
| rpl_semi_sync_master_wait_point | AFTER_COMMIT |
| rpl_semi_sync_slave_delay_master | OFF |
| rpl_semi_sync_slave_enabled | OFF |
| rpl_semi_sync_slave_kill_conn_timeout | 5 |
| rpl_semi_sync_slave_trace_level | 32 |
+---------------------------------------+--------------+
9 rows in set (0.006 sec)
MariaDB [test]> show global status like '%semi%';
+--------------------------------------------+-------+
| Variable_name | Value |
+--------------------------------------------+-------+
| Rpl_semi_sync_master_clients | 0 |
| Rpl_semi_sync_master_get_ack | 0 |
| Rpl_semi_sync_master_net_avg_wait_time | 0 |
| Rpl_semi_sync_master_net_wait_time | 0 |
| Rpl_semi_sync_master_net_waits | 0 |
| Rpl_semi_sync_master_no_times | 0 |
| Rpl_semi_sync_master_no_tx | 0 |
| Rpl_semi_sync_master_request_ack | 0 |
| Rpl_semi_sync_master_status | ON |
| Rpl_semi_sync_master_timefunc_failures | 0 |
| Rpl_semi_sync_master_tx_avg_wait_time | 0 |
| Rpl_semi_sync_master_tx_wait_time | 0 |
| Rpl_semi_sync_master_tx_waits | 0 |
| Rpl_semi_sync_master_wait_pos_backtraverse | 0 |
| Rpl_semi_sync_master_wait_sessions | 0 |
| Rpl_semi_sync_master_yes_tx | 0 |
| Rpl_semi_sync_slave_send_ack | 0 |
| Rpl_semi_sync_slave_status | OFF |
+--------------------------------------------+-------+
从：
yum -y install mariadb-server
[root@centos8-2 ~]$vim /etc/my.cnf.d/mariadb-server.cnf
##在[mysqld] 中添加如下字段
server-id=18 ##指定唯一服务器ID
plugin_load_add=semisync_slave
rpl_semi_sync_slave_enabled=ON
:保存退出
[root@centos8-2 ~]$systemctl start mariadb
[root@centos8-2 ~]$mysql
MariaDB [test]> show global variables like '%semi%';
ERROR 2006 (HY000): MySQL server has gone away
No connection. Trying to reconnect...
Connection id: 12
Current database: test

+---------------------------------------+--------------+
| Variable_name | Value |
+---------------------------------------+--------------+
| rpl_semi_sync_master_enabled | OFF |
| rpl_semi_sync_master_timeout | 10000 |
| rpl_semi_sync_master_trace_level | 32 |
| rpl_semi_sync_master_wait_no_slave | ON |
| rpl_semi_sync_master_wait_point | AFTER_COMMIT |
| rpl_semi_sync_slave_delay_master | OFF |
| rpl_semi_sync_slave_enabled | ON |
| rpl_semi_sync_slave_kill_conn_timeout | 5 |
| rpl_semi_sync_slave_trace_level | 32 |
+---------------------------------------+--------------+
MariaDB [test]> show global status like '%semi%';
+--------------------------------------------+-------+
| Variable_name | Value |
+--------------------------------------------+-------+
| Rpl_semi_sync_master_clients | 0 |
| Rpl_semi_sync_master_get_ack | 0 |
| Rpl_semi_sync_master_net_avg_wait_time | 0 |
| Rpl_semi_sync_master_net_wait_time | 0 |
| Rpl_semi_sync_master_net_waits | 0 |
| Rpl_semi_sync_master_no_times | 0 |
| Rpl_semi_sync_master_no_tx | 0 |
| Rpl_semi_sync_master_request_ack | 0 |
| Rpl_semi_sync_master_status | OFF |
| Rpl_semi_sync_master_timefunc_failures | 0 |
| Rpl_semi_sync_master_tx_avg_wait_time | 0 |
| Rpl_semi_sync_master_tx_wait_time | 0 |
| Rpl_semi_sync_master_tx_waits | 0 |
| Rpl_semi_sync_master_wait_pos_backtraverse | 0 |
| Rpl_semi_sync_master_wait_sessions | 0 |
| Rpl_semi_sync_master_yes_tx | 0 |
| Rpl_semi_sync_slave_send_ack | 0 |
| Rpl_semi_sync_slave_status | ON |
+--------------------------------------------+-------+
18 rows in set (0.001 sec)
主master上查看：
MariaDB [test]> show global status like '%semi%';
+--------------------------------------------+-------+
| Variable_name | Value |
+--------------------------------------------+-------+
| Rpl_semi_sync_master_clients | 1 | ##从一个节点 配置成功
| Rpl_semi_sync_master_get_ack | 0 |
| Rpl_semi_sync_master_net_avg_wait_time | 0 |
| Rpl_semi_sync_master_net_wait_time | 0 |
| Rpl_semi_sync_master_net_waits | 0 |
| Rpl_semi_sync_master_no_times | 0 |
| Rpl_semi_sync_master_no_tx | 0 |
| Rpl_semi_sync_master_request_ack | 0 |
| Rpl_semi_sync_master_status | ON |
| Rpl_semi_sync_master_timefunc_failures | 0 |
| Rpl_semi_sync_master_tx_avg_wait_time | 0 |
| Rpl_semi_sync_master_tx_wait_time | 0 |
| Rpl_semi_sync_master_tx_waits | 0 |
| Rpl_semi_sync_master_wait_pos_backtraverse | 0 |
| Rpl_semi_sync_master_wait_sessions | 0 |
| Rpl_semi_sync_master_yes_tx | 0 |
| Rpl_semi_sync_slave_send_ack | 0 |
| Rpl_semi_sync_slave_status | OFF |
+--------------------------------------------+-------+
18 rows in set (0.001 sec)
主上创建数据库，立即成功
MariaDB [test]> create database db2;
Query OK, 1 row affected (0.001 sec)
在所有slave节点实现，停止复制线程
MariaDB [test]> stop slave;
Query OK, 0 rows affected (0.005 sec)
在主master创建数据库，等待3S后显示成功（master配置文件定义的超时时间为3S）
MariaDB [test]> create database db3;
Query OK, 1 row affected (3.001 sec)
在从节点恢复复制线程
MariaDB [test]> start slave;
Query OK, 0 rows affected (0.001 sec)
在主master创建数据库，立即成功。
MariaDB [test]> create database db4;
Query OK, 1 row affected (0.001 sec)

在所有从节点停止同步线程，在主节点可也看到以下日志信息
[root@centos8-1 ~]$tail -f /var/log/mariadb/mariadb.log
2020-08-30 15:58:13 11 [Note] Start binlog_dump to slave_server(18), pos(mariadb-bin.000002, 1268)
2020-08-30 16:00:49 12 [Note] Start binlog_dump to slave_server(18), pos(mariadb-bin.000003, 344)
2020-08-30 16:00:49 12 [Note] Start semi-sync binlog_dump to slave (server_id: 18), pos(mariadb-bin.000003, 344)
2020-08-30 16:04:03 12 [Note] Stop semi-sync binlog_dump to slave (server_id: 18)
2020-08-30 16:04:27 10 [Warning] Timeout waiting for reply of binlog (file: mariadb-bin.000003, pos: 598), semi-sync up to file mariadb-bin.000003, position 471.
2020-08-30 16:04:27 10 [Note] Semi-sync replication switched OFF.
2020-08-30 16:05:50 14 [Note] Start binlog_dump to slave_server(18), pos(mariadb-bin.000003, 471)
2020-08-30 16:05:50 14 [Note] Start semi-sync binlog_dump to slave (server_id: 18), pos(mariadb-bin.000003, 471)
2020-08-30 16:05:50 6 [Note] Semi-sync replication switched ON with slave (server_id: 18) at (mariadb-bin.000003, 598)
2020-08-30 16:07:02 14 [Note] Stop semi-sync binlog_dump to slave (server_id: 18)

 

 

#### 5 MariaDB高可用方案MHA 准备四台新安装的服务器， IP地址分别为10.0.0.7 mha     10.0.0.8 master  10.0.0.18 slave1   10.0.0.28  slave2

MHA实验配置

manger节点
配置邮件服务
[root@centos7-1 ~]$vim /etc/mail.rc
set from=1185673631@qq.com
set smtp=smtp.qq.com
set smtp-auth-user=1185673631@qq.com
set smtp-auth-password=授权密码

[root@centos7-1 ~]$mail -s hello 1185673631@qq.com
test
.
EOT

安装包
[root@centos7-1 ~]$yum -y install mha4mysql-node-0.56-0.el6.noarch.rpm
[root@centos7-1 ~]$yum -y install mha4mysql-manager-0.56-0.el6.noarch.rpm
[root@centos7-1 ~]$ssh-keygen
[root@centos7-1 ~]$ssh-copy-id 10.0.0.7
[root@centos7-1 ~]$rsync -av .ssh 10.0.0.8:/root
[root@centos7-1 ~]$rsync -av .ssh 10.0.0.18:/root
[root@centos7-1 ~]$rsync -av .ssh 10.0.0.28:/root
[root@centos7-1 ~]$mkdir /etc/mastermha
[root@centos7-1 mastermha]$vim app1.cnf
[server default]
user=mhauser
password=tianze
manager_workdir=/data/mastermha/app1/
manager_log=/data/mastermha/app1/manager.log
remote_workdir=/data/mastermha/app1/
ssh_user=root
repl_user=repluser
repl_password=tianze
ping_interval=1
master_ip_failover_script=/usr/local/bin/master_ip_failover
report_script=/usr/local/bin/sendmail.sh
check_repl_delay=0
[server1]
hostname=10.0.0.8
[server2]
hostname=10.0.0.18
[server3]
hostname=10.0.0.28
candidate_master=1

[root@centos7-1 bin]$pwd
/usr/local/bin
[root@centos7-1 bin]$vim master_ip_failover

检查环境
[root@centos7-1 ~]$masterha_check_ssh --conf=/etc/mastermha/app1.cnf
Tue Sep 1 15:52:53 2020 - [info] All SSH connection tests passed successfully.

root@centos7-1 ~]$masterha_check_repl --conf=/etc/mastermha/app1.cnf
Checking the Status of the script.. OK
Tue Sep 1 15:53:46 2020 - [info] OK.
Tue Sep 1 15:53:46 2020 - [warning] shutdown_script is not defined.
Tue Sep 1 15:53:46 2020 - [info] Got exit code 0 (Not master dead).

MySQL Replication Health is OK.

启动MHA 默认前台运行
[root@centos7-1 ~]$nohup masterha_manager --conf=/etc/mastermha/app1.cnf &> /dev/null

查看状态
[root@centos7-1 ~]$masterha_check_status --conf=/etc/mastermha/app1.cnf
app1 (pid:11765) is running(0:PING_OK), master:10.0.0.8

排错日志
[root@centos7-1 ~]$cat /data/mastermha/app1/manager.log

 

master节点
[root@centos8-1 ~]$yum -y install mha4mysql-node-0.56-0.el6.noarch.rpm
[root@centos8-1 ~]$vim /etc/my.cnf.d/mariadb-server.cnf
##添加
[mysqld]
server-id=1
log-bin
skip_name_resolve=1
general_log

[root@centos8-1 ~]$systemctl start mariadb
[root@centos8-1 ~]$mysql
MariaDB [(none)]> show master logs;
+--------------------+-----------+
| Log_name | File_size |
+--------------------+-----------+
| mariadb-bin.000001 | 28204 |
| mariadb-bin.000002 | 344 |
+--------------------+-----------+
2 rows in set (0.000 sec)

MariaDB [(none)]> grant replication slave on *.* to repluser@'10.0.0.%' identified by 'tianze';
Query OK, 0 rows affected (0.000 sec)

MariaDB [(none)]> grant all on *.* to mhauser@'10.0.0.%' identified by 'tianze';
Query OK, 0 rows affected (0.000 sec)

MariaDB [(none)]> flush privileges;
Query OK, 0 rows affected (0.000 sec)

[root@centos8-1 ~]$ifconfig eth0:1 10.0.0.100/24

查看到健康性检查
[root@centos8-1 ~]$tail -f /var/lib/mysql/centos8-1.log
200901 15:59:07 16 Query SELECT 1 As Value
200901 15:59:08 16 Query SELECT 1 As Value
200901 15:59:09 16 Query SELECT 1 As Value
200901 15:59:10 16 Query SELECT 1 As Value
200901 15:59:11 16 Query SELECT 1 As Value
200901 15:59:12 16 Query SELECT 1 As Value
200901 15:59:13 16 Query SELECT 1 As Value
200901 15:59:14 16 Query SELECT 1 As Value
200901 15:59:15 16 Query SELECT 1 As Value
200901 15:59:16 16 Query SELECT 1 As Value
200901 15:59:17 16 Query SELECT 1 As Value
200901 15:59:18 16 Query SELECT 1 As Value

停止master 发现会自动把VIP迁移到其他从节点 提升为新主节点
[root@centos8-1 ~] systemctl stop mariadb

 

slave节点1
[root@centos8-2 ~]$yum -y install mha4mysql-node-0.56-0.el6.noarch.rpm
##添加
[mysqld]
server-id=2
log-bin
read_only
relay_log_purge=0
skip_name_resolve=1

[root@centos8-2 ~]$systemctl start mariadb
[root@centos8-2 ~]$mysql
MariaDB [(none)]> change master to master_host='10.0.0.8',master_user='repluser',master_password='tianze',master_log_file='mariadb-bin.000002',master_log_pos=344;
MariaDB [(none)]> start slave;
Query OK, 0 rows affected (0.001 sec)
MariaDB [(none)]> show slave statusG

 

slave节点2
[root@centos8-3 ~]$yum -y install mha4mysql-node-0.56-0.el6.noarch.rpm
##添加
[mysqld]
server-id=3
log-bin
read_only
relay_log_purge=0
skip_name_resolve=1

[root@centos8-3 ~]$systemctl start mariadb
[root@centos8-3 ~]$mysql
MariaDB [(none)]> change master to master_host='10.0.0.8',master_user='repluser',master_password='tianze',master_log_file='mariadb-bin.000002',master_log_pos=344;
MariaDB [(none)]> start slave;
Query OK, 0 rows affected (0.001 sec)
MariaDB [(none)]> show slave statusG

 

注：  MHA自动切换后会自动退出，然后修复节点重新加入集群中，需要删除如下文件，然后再开启MHA
删除文件
[root@centos7-1 ~]$rm -rf /data/mastermha/app1/app1.failover.complete

查看MHA集群状态
[root@centos7-1 ~]$masterha_check_status --conf=/etc/mastermha/app1.cnf

查看日志
tail -f /data/mastermha/app1/manager.log