MySQL PXC集群部署

安装 Percona-XtraDB-Cluster

架构：

三个节点：

pxc_node_0 30.0.0.196 
pxc_node_1 30.0.0.198 
pxc_node_2 30.0.0.199

防火墙：

-A INPUT -p tcp -m state --state NEW -m tcp --dport 3306 -j ACCEPT
-A INPUT -p tcp -m state --state NEW -m tcp --dport 4444 -j ACCEPT
-A INPUT -p tcp -m state --state NEW -m tcp --dport 4567 -j ACCEPT
-A INPUT -p tcp -m state --state NEW -m tcp --dport 4568 -j ACCEPT

软件包：

compat-readline5-5.2-17.4.x86_64.rpm
libev4-4.15-7.1.x86_64.rpm
numactl-2.0.9-2.el6.x86_64.rpm
percona-xtrabackup-24-2.4.12-1.el6.x86_64.rpm
Percona-XtraDB-Cluster-57-5.7.23-31.31.2.el6.x86_64.rpm
Percona-XtraDB-Cluster-client-57-5.7.23-31.31.2.el6.x86_64.rpm
Percona-XtraDB-Cluster-devel-57-5.7.23-31.31.2.el6.x86_64.rpm
Percona-XtraDB-Cluster-server-57-5.7.23-31.31.2.el6.x86_64.rpm
Percona-XtraDB-Cluster-shared-57-5.7.23-31.31.2.el6.x86_64.rpm
qpress-11-1.el6.x86_64.rpm
socat-1.7.2.4-1.el6.rf.x86_64.rpm

安装：

下载：

wget https://repo.percona.com/release/centos/6Server/RPMS/x86_64/Percona-XtraDB-Cluster-devel-57-5.7.23-31.31.2.el6.x86_64.rpm
wget https://repo.percona.com/release/centos/6Server/RPMS/x86_64/Percona-XtraDB-Cluster-server-57-5.7.23-31.31.2.el6.x86_64.rpm
wget https://repo.percona.com/release/centos/6Server/RPMS/x86_64/Percona-XtraDB-Cluster-57-5.7.23-31.31.2.el6.x86_64.rpm
wget https://repo.percona.com/release/centos/6Server/RPMS/x86_64/Percona-XtraDB-Cluster-client-57-5.7.23-31.31.2.el6.x86_64.rpm
wget https://repo.percona.com/release/centos/6Server/RPMS/x86_64/Percona-XtraDB-Cluster-shared-57-5.7.23-31.31.2.el6.x86_64.rpm
wget https://repo.percona.com/release/centos/6Server/RPMS/x86_64/percona-xtrabackup-24-2.4.12-1.el6.x86_64.rpm
wget https://repo.percona.com/release/centos/6Server/RPMS/x86_64/qpress-11-1.el6.x86_64.rpm
wget ftp://ftp.pbone.net/mirror/ftp5.gwdg.de/pub/opensuse/repositories/home:/viliampucik:/obs/RedHat_RHEL-6/x86_64/compat-readline5-5.2-17.4.x86_64.rpm
wget ftp://ftp.pbone.net/mirror/ftp.centos.org/6.9/os/x86_64/Packages/numactl-2.0.9-2.el6.x86_64.rpm
wget ftp://ftp.pbone.net/mirror/apt.sw.be/redhat/el6/en/x86_64/rpmforge/RPMS/socat-1.7.2.4-1.el6.rf.x86_64.rpm
wget ftp://ftp.pbone.net/mirror/ftp5.gwdg.de/pub/opensuse/repositories/home:/rudi_m:/devel-snap/CentOS_CentOS-6/x86_64/libev4-4.15-7.1.x86_64.rpm

安装：

rpm -ivh libev4-4.15-7.1.x86_64.rpm
rpm -ivh socat-1.7.2.4-1.el6.rf.x86_64.rpm
rpm -ivh numactl-2.0.9-2.el6.x86_64.rpm
rpm -ivh compat-readline5-5.2-17.4.x86_64.rpm
rpm -ivh percona-xtrabackup-24-2.4.8-1.el6.x86_64.rpm
rpm -ivh qpress-11-1.el6.x86_64.rpm
rpm -ivh Percona-XtraDB-Cluster-shared-57-5.7.19-29.22.3.el6.x86_64.rpm
rpm -ivh Percona-XtraDB-Cluster-client-57-5.7.19-29.22.3.el6.x86_64.rpm
rpm -ivh Percona-XtraDB-Cluster-57-5.7.19-29.22.3.el6.x86_64.rpm
rpm -ivh Percona-XtraDB-Cluster-server-57-5.7.19-29.22.3.el6.x86_64.rpm
rpm -ivh Percona-XtraDB-Cluster-devel-57-5.7.19-29.22.3.el6.x86_64.rpm 

配置：

配置文件：

vim /etc/my.cnf
[client]
port=3306
socket = /tmp/mysql.sock

[mysqld]
server-id=1
port=3306
datadir=/data/database/mysql
socket=/tmp/mysql.sock
log-error=/var/run/mysqld/mysqld.log
pid-file=/var/run/mysqld/mysqld.pid
user=mysql

# General
back_log=2000
connect_timeout=15
skip_name_resolve=ON
max_connections=5000
table_definition_cache=2000
table_open_cache=10000
metadata_locks_hash_instances=256
ssl=0
core_file

# Innodb
innodb_buffer_pool_size = 10240M
innodb_buffer_pool_instances=8
innodb_log_file_size = 1024M
innodb_log_buffer_size = 16M
innodb_lock_wait_timeout = 20
innodb_autoinc_lock_mode=2
innodb_read_io_threads = 5
innodb_write_io_threads = 5
innodb_thread_concurrency = 8
innodb_doublewrite=1
innodb_flush_log_at_trx_commit = 2
innodb_flush_method = 'O_DIRECT'
innodb-page-cleaners=8
innodb_purge_threads=4
innodb_lru_scan_depth=2048
innodb_io_capacity=8000
innodb_io_capacity_max=16000
innodb_adaptive_hash_index=OFF
innodb-change-buffering=none
innodb_flush_neighbors=0
innodb_max_dirty_pages_pct = 90
innodb_max_dirty_pages_pct_lwm = 10

# Binlog
relay-log=relay-1
binlog_format=ROW
enforce-gtid-consistency
gtid-mode=on
master-info-repository=TABLE
relay-log-info-repository=TABLE
binlog-checksum=NONE
log-bin
log_slave_updates
expire_logs_days=3
sync_binlog=1

# Monitoring
innodb_monitor_enable='%'
performance_schema=ON
performance_schema_instrument='%synch%=on'

# Galera
symbolic-links=0
explicit_defaults_for_timestamp=true
wsrep_provider=/usr/lib64/galera3/libgalera_smm.so
wsrep_cluster_address=gcomm://30.0.0.196,30.0.0.198,30.0.0.199
default_storage_engine=InnoDB
wsrep_slave_threads= 20
wsrep_log_conflicts
wsrep_cluster_name=pxc-cluster
wsrep_node_name=pxc3
wsrep_node_address=30.0.0.196
pxc_strict_mode=ENFORCING
wsrep_sst_method=xtrabackup-v2
wsrep_sst_auth="sstuser:s3cret"

差异化配置：

# 需要根据相应的节点配置
server-id=1
wsrep_node_name=pxc3
wsrep_node_address=30.0.0.196

配置数据目录：

mkdir -p /data/database/mysql
chown mysql:mysql -R /data/database/mysql

启动：

启动第一个节点

/etc/init.d/mysql bootstrap-pxc 

或者这样：

# 初始化
/usr/sbin/mysqld --defaults-file=/etc/my.cnf --initialize --datadir=/data/database/mysql --user=mysql 
# 启动
/usr/bin/mysqld_safe --defaults-file=/etc/my.cnf --wsrep-new-cluster &

查看密码

# root初始化密码被记录到log-error中
grep 'temporary password'  /var/run/mysqld/mysqld.log

修改root密码，并配置SST认证账号

mysql> CREATE USER 'sstuser'@'localhost' IDENTIFIED BY 's3cret';
mysql> GRANT RELOAD, LOCK TABLES, PROCESS, REPLICATION CLIENT ON *.* TO 'sstuser'@'localhost';
mysql> FLUSH PRIVILEGES;

启动其他节点

/etc/init.d/mysql start

各个加点都采用第一个几点修改的root密码登录

偷窥一下：

查看节点数：

show global status like 'wsrep_cluster_size';

查看集群状态：

show global status like 'wsrep%';

查看当前节点状态：

show global status like 'wsrep_cluster_status'; 

---

讲点小故事

重要的配置参数

• wsrep_provider：指定Galera库路径，没有这个库，节点只是一个MySQL单实例，无法参与pxc集群

  • Debian or Ubuntu: /usr/lib/libgalera_smm.so
  • Red Hat or CentOS: /usr/lib64/galera3/libgalera_smm.so

• wsrep_cluster_name：集群名称，所有节点相同

• wsrep_cluster_address：集群所有节点IP地址列表

• wsrep_node_name：节点名称，默认为主机名称

• wsrep_node_address：节点IP地址

• wsrep_sst_method：指定SST方法，推荐wsrep_sst_method=xtrabackup-v2

• wsrep_sst_auth：SST认证账号密码

  • <sst_user>:<sst_pass>
  • 当第一个节点启动后手动创建该账号密码，并授予相应的权限

• pxc_strict_mode：PXC严格控制模式，

一些名词解释

• WS：write set写数据集，写/更新事务

• IST：Incremental State Transfer 增量同步

• SST：State Snapshot Transfe 全量同步。

  • SST支持的方法有：mysqldump，rsync ，xtrabackup
  • mysqldump，rsync同步需要READ LOCK， (SST applies FLUSH TABLES WITH READ LOCK command)
  • xtrabackup 在整个同步数据过程中不需要READ LOCK
  • 配置参数：wsrep_sst_method=xtrabackup-v2

• UUID：节点状态改变及顺序的唯一标识

• GTID：Global Transaction ID，由UUID和sequence number偏移量组成。wsrep api中定义的集群内部全局事务id，用于记录集群中发生状态改变的唯一标识以及队列中的偏移量。

• WSRWP API：在DBMS库和wsrep provider之间提供接口

集群端口说明

• 3306：数据库对外提供服务的端口

• 4444：全量数据传输SST，集群数据同步端口，全量同步，新节点加入时启用该端口接受数据（DONOR节点接受新节点的请求后会启动一个随机端口主动来连接4444端口）

• 4567：集群节点间相互通信的端口

• 4568：增量数据同步IST，节点下线、重启后启用该端口接受数据（DONOR节点接受新节点的请求后会启动一个随机端口主动来连接4568端口）