pxc群集搭建
1.环境
Percona-XtraDB 5.7.22-22-29.26-log
percona-xtrabackup-24-2.4.12
192.168.99.210:3101(第一节点)
192.168.99.211:3101
192.168.99.212:3101
2.安装
- 安装包获取(根据操作系统类型获取不同安装包):
wget https://www.percona.com/downloads/Percona-XtraDB-Cluster-LATEST/Percona-XtraDB-Cluster-5.7.22-29.26/binary/tarball/Percona-XtraDB-Cluster-5.7.22-rel22-29.26.1.Linux.x86_64.ssl102.tar.gz
wget https://www.percona.com/downloads/XtraBackup/Percona-XtraBackup-2.4.12/binary/redhat/7/x86_64/percona-xtrabackup-24-2.4.12-1.el7.x86_64.rpm - 所有节点上做以下操作:
\安装依赖包
yum install -y git scons gcc gcc-c++ openssl check cmake bison boost-devel asio-devel libaio-devel ncurses-devel readline-devel pam-devel socat
\安装xtrabackup
yum localinstall percona-xtrabackup-24-2.4.12-1.el7.x86_64.rpm
tar -xzvf tarball/Percona-XtraDB-Cluster-5.7.22-rel22-29.26.1.Linux.x86_64.ssl102.tar.gz /opt/
ln -s /opt/Percona-XtraDB-Cluster-5.7.22-rel22-29.26.1.Linux.x86_64.ssl102/ /usr/local/pxc
3.初始化第一节点
mkdir /data/pxc/mysql3101/{data,logs,tmp}
chown -R mysql:mysql /data/pxc/*
ln -s /data/pxc/mysql3101/ /3101
- 修改my_pxc.cnf
如需要全配置文件可在github.com/k2865下载
loose_wsrep_cluster_name=pxc_wenyz
loose_wsrep_cluster_address=gcomm://192.168.99.210,192.168.99.211,192.168.99.212
loose_wsrep_node_address=192.168.99.210
#//建议将原SST使用的4444端口改为其他端口,因为4444是msblast蠕虫病毒使用端口,在有云安全产品下可能会被拦截.
wsrep_sst_receive_address =192.168.99.210:4588
loose_wsrep_provider=/usr/local/pxc/lib/libgalera_smm.so
#wsrep_sst_method=reync
loose_wsrep_sst_method=xtrabackup-v2
loose_wsrep_sst_auth=sst:zstxxx
- 初始化实例和启动
初始化:/usr/local/pxc/bin/mysqld --defaults-file=/3101/my_pxc.cnf --initialize-insecure
启动:
/usr/local/pxc/bin/mysqld --defaults-file=/3101/my_pxc.cnf --wsrep-new-cluster&
- 创建sst账号:
\登录mysql
GRANT RELOAD, PROCESS, SUPER, REPLICATION CLIENT ON *.* TO 'sst'@'%' identified by 'zstxxx';
4.其他节点
其他节点不用初始化,直接启动就可以.但启动过程中会遇到很多报错,需要逐一排除.
/usr/local/pxc/bin/mysqld --defaults-file=/3101/my_pxc.cnf&
- 成功加入的过程(第二节点)
刚启动时监听端口状态:
#netstat -nltp
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 0.0.0.0:4588(SST) 0.0.0.0:* LISTEN 22451/socat
tcp 0 0 0.0.0.0:3508 0.0.0.0:* LISTEN 16289/mysqld
tcp 0 0 0.0.0.0:4567(PXC通信) 0.0.0.0:* LISTEN 21978/mysqld
[root@db212_08:58:42 /3101]
\节点初始化完成后的端监听状态:
#netstat -nltp
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 0.0.0.0:4567 0.0.0.0:* LISTEN 22431/mysqld
tcp6 0 0 :::3101 (mysql) :::* LISTEN 22431/mysqld
- 登录节点查看状态:
root@localhost [(none)]>show global status like 'wsrep_%';
+----------------------------------+------------------------------------------------------------------+
| Variable_name | Value |
+----------------------------------+------------------------------------------------------------------+
| wsrep_local_state_uuid | 8c3106bd-9a40-11e8-8088-822c237073c1 |
| wsrep_protocol_version | 8 |
| wsrep_last_applied | 6 |
| wsrep_last_committed | 6 |
| wsrep_replicated | 1 |
| wsrep_replicated_bytes | 400 |
| wsrep_repl_keys | 2 |
| wsrep_repl_keys_bytes | 40 |
| wsrep_repl_data_bytes | 295 |
| wsrep_repl_other_bytes | 0 |
| wsrep_received | 40 |
| wsrep_received_bytes | 5939 |
| wsrep_local_commits | 0 |
| wsrep_local_cert_failures | 0 |
| wsrep_local_replays | 0 |
| wsrep_local_send_queue | 0 |
| wsrep_local_send_queue_max | 2 |
| wsrep_local_send_queue_min | 0 |
| wsrep_local_send_queue_avg | 0.250000 |
| wsrep_local_recv_queue | 0 |
| wsrep_local_recv_queue_max | 1 |
| wsrep_local_recv_queue_min | 0 |
| wsrep_local_recv_queue_avg | 0.000000 |
| wsrep_local_cached_downto | 5 |
| wsrep_flow_control_paused_ns | 0 |
| wsrep_flow_control_paused | 0.000000 |
| wsrep_flow_control_sent | 0 |
| wsrep_flow_control_recv | 0 |
| wsrep_flow_control_interval | [ 173, 173 ] |
| wsrep_flow_control_interval_low | 173 |
| wsrep_flow_control_interval_high | 173 |
| wsrep_flow_control_status | OFF |
| wsrep_cert_deps_distance | 1.000000 |
| wsrep_apply_oooe | 0.000000 |
| wsrep_apply_oool | 0.000000 |
| wsrep_apply_window | 1.000000 |
| wsrep_commit_oooe | 0.000000 |
| wsrep_commit_oool | 0.000000 |
| wsrep_commit_window | 1.000000 |
| wsrep_local_state | 4 |
| wsrep_local_state_comment | Synced |
| wsrep_cert_index_size | 2 |
| wsrep_cert_bucket_count | 22 |
| wsrep_gcache_pool_size | 336152 |
| wsrep_causal_reads | 0 |
| wsrep_cert_interval | 0.000000 |
| wsrep_ist_receive_status | |
| wsrep_ist_receive_seqno_start | 0 |
| wsrep_ist_receive_seqno_current | 0 |
| wsrep_ist_receive_seqno_end | 0 |
| wsrep_incoming_addresses | 192.168.99.212:3101,192.168.99.211:3101,192.168.99.210:3101 |
| wsrep_desync_count | 0 |
| wsrep_evs_delayed | 2c3c416a-9aa6-11e8-a518-77bd362692b0:tcp://192.168.99.212:4567:1 |
| wsrep_evs_evict_list | |
| wsrep_evs_repl_latency | 0/0/0/0/0 |
| wsrep_evs_state | OPERATIONAL |
| wsrep_gcomm_uuid | 8d6121e1-9a42-11e8-98e7-6b000d25448a |
| wsrep_cluster_conf_id | 29 |
| wsrep_cluster_size | 3 |
| wsrep_cluster_state_uuid | 8c3106bd-9a40-11e8-8088-822c237073c1 |
| wsrep_cluster_status | Primary |
| wsrep_connected | ON |
| wsrep_local_bf_aborts | 0 |
| wsrep_local_index | 1 |
| wsrep_provider_name | Galera |
| wsrep_provider_vendor | Codership Oy <info@codership.com> |
| wsrep_provider_version | 3.26(r) |
| wsrep_ready | ON |
+----------------------------------+------------------------------------------------------------------+
68 rows in set (0.00 sec)
root@localhost [(none)]>
5.遇到过的报错
- [ERROR] socat not found
- 解决办法:yum install -y socat
29.26.1.Linux.x86_64.ssl102/bin:/usr/local/mysql/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/root/bin)
2018-08-07T04:25:23.504726Z WSREP_SST: [ERROR] ******************* FATAL ERROR **********************
2018-08-07T04:25:23.505847Z WSREP_SST: [ERROR] socat not found in path: /usr/sbin:/sbin:/opt/Percona-XtraDB-Cluster-5.7.22-rel22-29.26.1.Linux.x86_64.ssl102/bin:/usr/local/mysql/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/root/bin
2018-08-07T04:25:23.506926Z WSREP_SST: [ERROR] ******************************************************
2018-08-07T04:25:23.507286Z 0 [ERROR] WSREP: Failed to read 'ready <addr>' from: wsrep_sst_xtrabackup-v2 --role 'joiner' --address '192.168.99.210' --datadir '/3101/data/' --defaults-file '/3101/my_pxc.cnf' --defaults-group-suffix '' --parent '20904' ''
Read: '(null)'
2018-08-07T04:25:23.507316Z 0 [ERROR] WSREP: Process completed with error: wsrep_sst_xtrabackup-v2 --role 'joiner' --address '192.168.99.210' --datadir '/3101/data/' --defaults-file '/3101/my_pxc.cnf' --defaults-group-suffix '' --parent '20904' '' : 2 (No such file or directory)
2018-08-07T04:25:23.507360Z 1 [ERROR] WSREP: Failed to prepare for 'xtrabackup-v2' SST. Unrecoverable.
2018-08-07T04:25:23.507371Z 1 [ERROR] Aborting
- Possible timeout in receving first data from donor in gtid/keyring stage
在donor的error.log里查看发现无法连接192.168.99.211:4444 将4444改为其他端口即可解决
2018-08-07T04:50:52.153284Z 0 [Note] WSREP: gcomm: closing backend
2018-08-07T04:50:52.758009Z WSREP_SST: [ERROR] ******************* FATAL ERROR **********************
2018-08-07T04:50:52.759154Z WSREP_SST: [ERROR] Possible timeout in receving first data from donor in gtid/keyring stage
2018-08-07T04:50:52.760273Z WSREP_SST: [ERROR] ******************************************************
2018-08-07T04:50:52.761583Z WSREP_SST: [ERROR] Cleanup after exit with status:32
2018-08-07T04:50:54.579876Z 0 [Note] WSREP: (756ee09e, 'tcp://0.0.0.0:4567') turning message relay requesting off
2018-08-07T04:50:55.579830Z 0 [Note] WSREP: (756ee09e, 'tcp://0.0.0.0:4567') connection to peer b83607a8 with addr tcp://192.168.99.210:4567 timed out, no messages seen in PT3S (gmcast.peer_timeout)
2018-08-07T04:50:55.579922Z 0 [Note] WSREP: (756ee09e, 'tcp://0.0.0.0:4567') turning message relay requesting on, nonlive peers: tcp://192.168.99.210:4567
2018-08-07T04:50:57.079838Z 0 [Note] WSREP: (756ee09e, 'tcp://0.0.0.0:4567') reconnecting to b83607a8 (tcp://192.168.99.210:4567), attempt 0
2018-08-07T04:50:57.653421Z 0 [Note] WSREP: declaring node with index 1 suspected, timeout PT5S (evs.suspect_timeout)
2018-08-07T04:50:57.653448Z 0 [Note] WSREP: evs::proto(756ee09e, LEAVING, view_id(REG,756ee09e,20)) suspecting node: b83607a8
2018-08-07T04:50:57.653457Z 0 [Note] WSREP: evs::proto(756ee09e, LEAVING, view_id(REG,756ee09e,20)) suspected node without join message, declaring inactive
2018-08-07T04:50:57.653484Z 0 [Note] WSREP: Current view of cluster as seen by this node
view (view_id(NON_PRIM,756ee09e,20)
- Possible timeout in receving first data from donor in gtid/keyring
解决办法 :yum install -y git scons gcc gcc-c++ openssl check cmake bison boost-devel asio-devel libaio-devel ncurses-devel readline-devel pam-devel socat
018-08-07T12:05:49.545630Z 0 [Warning] WSREP: Member 2.0 (db212) requested state transfer from '*any*', but it is impossible to select State Transfer donor: Resource temporarily unavailable
2018-08-07T12:05:50.313182Z WSREP_SST: [ERROR] ******************* FATAL ERROR **********************
2018-08-07T12:05:50.314371Z WSREP_SST: [ERROR] Possible timeout in receving first data from donor in gtid/keyring stage
2018-08-07T12:05:50.315523Z WSREP_SST: [ERROR] ******************************************************
2018-08-07T12:05:50.316914Z WSREP_SST: [ERROR] Cleanup after exit with status:32
2018-08-07T12:05:50.326100Z 0 [ERROR] WSREP: Process completed with error: wsrep_sst_xtrabackup-v2 --role 'joiner' --address '192.168.99.212:4588' --datadir '/3101/data/' --defaults-file '/3101/my_pxc.cnf' --defaults-group-suffix '' --parent '17660' '' : 32 (Broken pipe)
2018-08-07T12:05:50.326134Z 0 [ERROR] WSREP: Failed to read uuid:seqno from joiner script.
2018-08-07T12:05:50.326146Z 0 [ERROR] WSREP: SST script aborted with error 32 (Broken pipe)
2018-08-07T12:05:50.326182Z 0 [ERROR] WSREP: SST failed: 32 (Broken pipe)
2018-08-07T12:05:50.326202Z 0 [ERROR] Aborting
2018-08-07T12:05:50.326207Z 0 [Note] WSREP: Signalling cancellation of the SST request.
2018-08-07T12:05:50.326239Z 0 [Note] WSREP: SST request was cancelled
2018-08-07T12:05:50.546015Z 0 [Warning] WSREP: Member 2.0 (db212) requested state transfer from '*any*', but it is impossible to select State Transfer donor: Resource temporarily unavailable
2018-08-07T12:05:51.546468Z 0 [Warning] WSREP: Member 2.0 (db212) requested state transfer from '*any*', but it is impossible to select State Transfer donor: Resource temporarily unavailable
2018-08-07T12:05:52.546924Z 0 [Warning] WSREP: Member 2.0 (db212) requested state transfer from '*any*', but it is impossible to select State Transfer donor: Resource temporarily unavailable
2018-08-07T12:05:53.547325Z 0 [Warning] WSREP: Member 2.0 (db212) requested state transfer from '*any*', but it is impossible to select State Transfer donor: Resource temporarily unavailable
2018-08-07T12:05:54.547722Z 0 [Warning] WSREP: Member 2.0 (db212) requested state transfer from '*any*', but it is impossible to select State Transfer donor: Resource temporarily unavailable
2018-08-07T12:05:55.548121Z 0 [Warning] WSREP: Member 2.0 (db212) requested state transfer from '*any*', but it is impossible to select State Transfer donor: Resource temporarily unavailable
2018-08-07T12:05:56.548496Z 0 [Warning] WSREP: Member 2.0 (db212) requested state transfer from '*any*', but it is impossible to select State Transfer donor: Resource temporarily unavailable
2018-08-07T12:05:57.548894Z 0 [Warning] WSREP: Member 2.0 (db212) requested state transfer from '*any*', but it is impossible to select State Transfer donor: Resource temporarily unavailable
2018-08-07T12:05:58.549273Z 0 [Warning] WSREP: Member 2.0 (db212) requested state transfer from '*any*', but it is impossible to select State Transfer donor: Resource temporarily unavailable
2018-08-07T12:05:59.549743Z 0 [Warning] WSREP: Member 2.0 (db212) requested state transfer from '*any*', but it is impossible to select State Transfer donor: Resource temporarily unavailable
2018-08-07T12:06:00.550210Z 0 [Warning] WSREP: Member 2.0 (db212) requested state transfer from '*any*', but it is impossible to select State Transfer donor: Resource temporarily unavailable