1概述
配置了四个Linux HA+ Oracle数据库集群。其配置信息如下:
1.1店会系统&FC-arap系统集群
集群资源组名称 |
ac_group |
fc_group |
集群资源文件挂载点 |
/ac-oradata |
/fc-oradata |
集群服务IP地址 |
fmpracora-vip/ 10.0.101.11 |
fmprfcora-vip/ 10.0.101.12 |
集群Listener |
cvs_lsn |
arap2_lsn |
集群数据库 |
cvsdb |
arap2db |
优先运行的节点名称 |
fmpracora-priv |
fmprfcora-priv |
优先运行的节点IP |
10.0.101.9 |
10.0.101.10 |
集群fence通讯IP |
10.0.202.15 |
10.0.202.19 |
1.2订购-营收&主档-预算检讨-基盘集群
集群资源组名称 |
ma_group |
po_group |
集群资源文件挂载点 |
/ma-oradata |
/po-oradata |
集群服务IP地址 |
fmprmaora-vip/ 10.0.101.7 |
fmprpoora-vip/ 10.0.101.8 |
集群Listener |
arpdb_lsn |
yingshou_lsn |
集群数据库 |
master_db, arpdb_db |
dingoudb, yingshou_db |
优先运行的节点名称 |
fmprmaora-priv |
fmprpoora-priv |
优先运行的节点IP |
10.0.101.5 |
10.0.101.6 |
集群fence通讯IP |
10.0.202.16 |
10.0.202.20 |
1.3IDC-SCM系统集群
集群资源组名称 |
idc_group |
集群资源文件挂载点 |
/idc-oradata |
集群服务IP地址 |
fmpridcora-vip/ 10.0.101.4 |
集群Listener |
arpdb_lsn |
集群数据库 |
fmidcdb |
优先运行的节点名称 |
fmpridcora-priv |
优先运行的节点IP |
10.0.101.4 |
集群fence通讯IP |
10.0.202.17 |
集群另外一个节点是10.0.101.3。
1.4JDE系统&JDE测试系统集群
集群资源组名称 |
pr_group |
dr_group |
集群资源文件挂载点 |
/jdepr-oradata |
/jdedr-oradata |
集群服务IP地址 |
fmprjdeora-vip/ 10.0.101. 15 |
fmdrjdeora-vip/ 10.0.101. 16 |
集群Listener |
jde_lsn |
jdedbdr_lsn |
集群数据库 |
jde_db |
jdedr_db |
优先运行的节点名称 |
fmprjdeora-priv |
fmdrjdeora-priv |
优先运行的节点IP |
10.0.101.13 |
10.0.101.14 |
集群fence通讯IP |
10.0.202.18 |
10.0.202.22 |
2安装配置手册
RHEL 6.8 HA和RHEL 7.2 HA的安装过程不相同,但是集群的配置过程相同。我们将软件安装过程分开,配置过程放在一起描述。
2.1 . RHEL 6.8 HA软件安装
将RHEL 6.8 ISO文件挂载到/software/rh68目录。编辑/etc/yum.conf文件。
2.1.1 编辑/etc/yum.conf
[local]
name=local
baseurl=file:///software/rh68
enabled=1
gpgcheck=1
gpgkey=file:///software/rh68/RPM-GPG-KEY-redhat-release
[HighAvailability]
name=HighAvailability
baseurl=file:///software/rh68/HighAvailability
enabled=1
gpgcheck=0
2.1.2 安装软件
# yum install pcs pacemaker cman fence-agents
从RHEL 6.5以后,RHEL的HA通过pcs来管理,不再通过rgmanager管理。只有RHEL 6.8才有Oracle的Agent。
2.1.3 禁用和停止主机防火墙服务
禁用防火墙
#chkconfig iptables off
#chkconfig ip6tables off
停止防火墙服务
#service iptables stop
#service ip6tables off
For TCP: Ports 2224, 3121, 21064
For UDP: Ports, 5405
2.1.4 启动和配置服务pcsd
# chkconfig pcsd on
# service pcsd start
2.1.5 重置hacluster密码
hacluster是pcs 管理员账号,建议集群中的服务器设置相同的密码。
# passwd hacluster (hacluster1234)
更改用户 hacluster 的密码 。
新的 密码:
无效的密码: 密码包含用户名在某些地方
重新输入新的 密码:
passwd:所有的身份验证令牌已经成功更新。
2.2RHEL 7.2 HA软件安装
2.2.1 /etc/yum.conf
[HighAvailability]
name=HighAvailability
baseurl=file:///software/rh72/addons/HighAvailability
enabled=1
gpgcheck=0
[local]
name=local
baseurl=file:///software/rh72/
enabled=1
gpgcheck=1
gpgkey=file:///software/rh72/RPM-GPG-KEY-redhat-release
2.2.2 安装软件
# yum install pcs fence-agents-all
只有RHEL 7.2才有Oracle的HA Agent, RHEL 7.0和7.1都没有Oracle的HA Agent。在RHEL 7.2,将不再使用cman服务.
2.2.3 启动和配置服务pcsd
[root@fmprfcdb ~]# systemctl start pcsd.service
[root@fmprfcdb ~]# systemctl enable pcsd.service
ln -s '/usr/lib/systemd/system/pcsd.service' '/etc/systemd/system/multi-user.target.wants/pcsd.service'
检查pcsd服务能随机自动重新启动
[root@fmpracora dinxin]# systemctl list-unit-files |grep pcs
pcsd.service enabled
rpcsvcgssd.service static
2.2.4 禁用和停止主机防火墙服务
#systemctl stop firewalld
2.2.5 重置hacluster密码
hacluster是pcs 管理员账号,建议集群中的服务器设置相同的密码。
# passwd hacluster (hacluster1234)
更改用户 hacluster 的密码 。
新的 密码:
无效的密码: 密码包含用户名在某些地方
重新输入新的 密码:
passwd:所有的身份验证令牌已经成功更新。
2.3配置集群
RHEL 6.8和RHEL 7.2配置集群的步骤相同。将在以下介绍。
2.3.1 编辑/etc/hosts
[root@fmprpoora oracle]# cat /etc/hosts
127.0.0.1localhost localhost.localdomain localhost4 localhost4.localdomain4
::1localhost localhost.localdomain localhost6 localhost6.localdomain6
10.0.101.6 fmprmaora
10.0.101.5 fmprpoora
10.0.200.16 fmprpoora-priv
10.0.200.20 fmprmaora-priv
10.0.101.7fmprpoora-vip
10.0.101.8fmprmaora-vip
其中fmprpoora-priv,fmprmaora-priv为心跳IP地址对应的主机名,将用作集群间的通讯。
2.3.2 授权hacluster 访问集群中的节点
# pcs cluster auth fmprfcdb-priv fmpracdb-priv
Username: hacluster
Password:
fmprfcdb-priv: Authorized
fmpracdb-priv: Authorized
授权的令牌保存在以下目录中。
[root@fmprpoora oracle]# cat /var/lib/pcsd/tokens
{
"format_version": 2,
"data_version": 3,
"tokens": {
"fmprmaora-priv": "a2a19af3-584e-4f88-873b-c3bee27d137c",
"fmprpoora-priv": "33169580-5ba5-4cb3-9a1c-a4a87fd3d226"
}
2.3.3 创建cluster
[root@fmprfcora pcsd]# pcs cluster setup --start --name fmprfcac_cluster fmprfcora-priv fmpracora-priv
Shutting down pacemaker/corosync services...
Redirecting to /bin/systemctl stop pacemaker.service
Redirecting to /bin/systemctl stop corosync.service
Killing any remaining services...
Removing all cluster configuration files...
fmprfcora-priv: Succeeded
fmpracora-priv: Succeeded
Starting cluster on nodes: fmprfcora-priv, fmpracora-priv...
fmpracora-priv: Starting Cluster...
fmprfcora-priv: Starting Cluster...
Synchronizing pcsd certificates on nodes fmprfcora-priv, fmpracora-priv...
fmprfcora-priv: Success
fmpracora-priv: Success
Restaring pcsd on the nodes in order to reload the certificates...
fmprfcora-priv: Success
fmpracora-priv: Success
[root@fmprfcora pcsd]#
2.3.4 配置集群服务随主机启动而自动启动
[root@fmprfcora pcsd]# pcs cluster enable --all
fmprfcora-priv: Cluster Enabled
fmpracora-priv: Cluster Enabled
2.3.5 检查cluster服务状态
[root@fmpracdb pcsd]# pcs cluster status
Cluster Status:
Last updated: Sun Aug 7 11:07:59 2016 Last change: Sun Aug 7 11:06:44 2016 by hacluster via crmd on fmpracora-priv
Stack: corosync
Current DC: fmpracora-priv (version 1.1.13-10.el7-44eb2dd) - partition with quorum
2 nodes and 0 resources configured
Online: [ fmpracora-priv fmprfcora-priv ]
PCSD Status:
fmprfcora-priv: Online
fmpracora-priv: Online
2.3.6 配置集群的fence设备
#pcs stonith create imm-fmprjdeora fence_ipmilan action=reboot auth=password ipaddr=10.0.202.18 lanplus=1 login=USERID passwd=PASSW0RD delay=120cipher=1 pcmk_host_list=fmprjdeora-priv pcmk_host_check=static-list power_timeout=60
#pcs stonith create imm-fmdrjdeora fence_ipmilan action=reboot auth=password ipaddr=10.0.202.22 lanplus=1 login=USERID passwd=PASSW0RD cipher=1 pcmk_host_list=fmdrjdeora-priv pcmk_host_check=static-list power_timeout=60
查看当前fence设备配置的命令如下:
[root@fmprpoora oracle]# pcs stonith show --full
Resource: imm-fmprmaora (class=stonith type=fence_ipmilan)
Attributes: action=reboot auth=password ipaddr=10.0.202.20 lanplus=1 login=USERID passwd=PASSW0RD cipher=1 pcmk_host_list=fmprmaora-priv pcmk_host_check=static-list power_timeout=60
Operations: monitor interval=60s (imm-fmprmaora-monitor-interval-60s)
Resource: imm-fmprpoora (class=stonith type=fence_ipmilan)
Attributes: action=reboot auth=password ipaddr=10.0.202.16 lanplus=1 login=USERID passwd=PASSW0RD delay=5 cipher=1 pcmk_host_list=fmprpoora-priv pcmk_host_check=static-list power_timeout=60 debug=/tmp/fmprpoora.log
Operations: monitor interval=60s (imm-fmprpoora-monitor-interval-60s)
[root@fmprpoora oracle]#
2.3.7 测试fence
检查能否使用fence_ipmilan命令查看fence设备的状态
[root@fmpridcora oracle]# fence_ipmilan -a 10.0.202.22 -P -l USERID -p PASSW0RD -o status -v
Executing: /usr/bin/ipmitool -I lanplus -H 10.0.202.22 -U USERID -P PASSW0RD -p 623 -L ADMINISTRATOR chassis power status
0 Chassis Power is on
Status: ON
测试能否使用命令行将节点fence,fence之后的设备将自动重新启动
root@fmprpoora ~]# pcs stonith fence fmprmaora-priv
Node: fmprmaora-priv fenced
[root@fmprmaora ~]# pcs stonith fence fmprpoora-priv
Node: fmprpoora-priv fenced
2.3.8 创建浮动IP
创建浮动IP地址资源
[root@fmprfcora pcsd]# pcs resource create fmprfcora-vip ocf:heartbeat:IPaddr2 ip=10.0.101.12 cidr_netmask=24 op monitor interval=30s
[root@fmprfcora pcsd]# pcs resource create fmpracora-vip ocf:heartbeat:IPaddr2 ip=10.0.101.11 cidr_netmask=24 op monitor interval=30s
[root@fmprfcora pcsd]# pcs resource show --full
Resource: fmprfcora-vip (class=ocf provider=heartbeat type=IPaddr2)
Attributes: ip=10.0.101.12 cidr_netmask=24
Operations: start interval=0s timeout=20s (fmprfcora-vip-start-interval-0s)
stop interval=0s timeout=20s (fmprfcora-vip-stop-interval-0s)
monitor interval=30s (fmprfcora-vip-monitor-interval-30s)
Resource: fmpracora-vip (class=ocf provider=heartbeat type=IPaddr2)
Attributes: ip=10.0.101.11 cidr_netmask=24
Operations: start interval=0s timeout=20s (fmpracora-vip-start-interval-0s)
stop interval=0s timeout=20s (fmpracora-vip-stop-interval-0s)
monitor interval=30s (fmpracora-vip-monitor-interval-30s)
2.3.9 创建文件系统资源
文件系统和挂载点需要已经创建好,并在集群中的各节点依次挂载完成。接下来在集群的文件系统资源。
[root@fmpracdb pcsd]# pcs resource create ac_fs Filesystem
>device="/dev/mapper/acvg-aclv" directory="/ac-oradata" fstype="ext4"
[root@fmpracdb pcsd]# pcs resource create fc_fs Filesystem
>device="/dev/mapper/fcvg-fclv" directory="/fc-oradata" fstype="ext4"
显示文件系统资源的配置
[root@fmpracora dinxin]# pcs resource show ac_fs
Resource: ac_fs (class=ocf provider=heartbeat type=Filesystem)
Attributes: device=/dev/mapper/acvg-aclv directory=/ac-oradata fstype=ext4
Operations: start interval=0s timeout=60 (ac_fs-start-interval-0s)
stop interval=0s timeout=60 (ac_fs-stop-interval-0s)
monitor interval=20 timeout=40 (ac_fs-monitor-interval-20)
[root@fmpracora dinxin]# pcs resource show fc_fs
Resource: fc_fs (class=ocf provider=heartbeat type=Filesystem)
Attributes: device=/dev/mapper/fcvg-fclv directory=/fc-oradata fstype=ext4
Operations: start interval=0s timeout=60 (fc_fs-start-interval-0s)
stop interval=0s timeout=60 (fc_fs-stop-interval-0s)
monitor interval=20 timeout=40 (fc_fs-monitor-interval-20)
2.3.10 创建数据库lsnr和db资源
确认当前pcs版本支持Oracle数据库。以下表示支持oracle数据库
[root@fmpracora dinxin]# pcs resource list |grep ora
ocf:heartbeat:oracle - Manages an Oracle Database instance
ocf:heartbeat:oralsnr - Manages an Oracle TNS listener
确认oracle数据库和listener已经创建好,并且手工能够在集群中的节点正常启动到open状态。
确保能解析数据库sid,
[oracle@fmpracora admin]$ tnsping cvs
TNS Ping Utility for Linux: Version 11.2.0.4.0 - Production on 23-AUG-2016 13:51:06
Copyright (c) 1997, 2013, Oracle. All rights reserved.
Used parameter files:
Used TNSNAMES adapter to resolve the alias
Attempting to contact (DESCRIPTION = (ADDRESS = (PROTOCOL = TCP)(HOST = fmpracora-vip)(PORT = 1526)) (CONNECT_DATA = (SERVER = DEDICATED) (SERVICE_NAME = cvs)))
OK (0 msec)
[oracle@fmpracora admin]$ tnsping arap2
TNS Ping Utility for Linux: Version 11.2.0.4.0 - Production on 23-AUG-2016 13:51:10
Copyright (c) 1997, 2013, Oracle. All rights reserved.
Used parameter files:
Used TNSNAMES adapter to resolve the alias
Attempting to contact (DESCRIPTION = (ADDRESS = (PROTOCOL = TCP)(HOST = fmprfcora-vip)(PORT = 1521)) (CONNECT_DATA = (SERVER = DEDICATED) (SERVICE_NAME = arap2)))
OK (0 msec)
当前的oracle listener.ora和tnsnames.ora文件如下:
CVS =
(DESCRIPTION =
(ADDRESS = (PROTOCOL = TCP)(HOST = fmpracora-vip)(PORT = 1526))
(CONNECT_DATA =
(SERVER = DEDICATED)
(SERVICE_NAME = cvs)
)
)
ARAP2 =
(DESCRIPTION =
(ADDRESS = (PROTOCOL = TCP)(HOST = fmprfcora-vip)(PORT = 1521))
(CONNECT_DATA =
(SERVER = DEDICATED)
(SERVICE_NAME = arap2)
)
)
[oracle@fmpracora admin]$ cat listener.ora
LSNR_arap2=
(ADDRESS_LIST =
(ADDRESS = (PROTOCOL = TCP)(HOST = fmprfcora-vip)(PORT = 1521))
)
SECURE_REGISTER_LSNR_arap2 = (TCP)
LSNR_cvs=
(ADDRESS_LIST =
(ADDRESS = (PROTOCOL = TCP)(HOST = fmpracora-vip)(PORT = 1526))
)
SECURE_REGISTER_LSNR_cvs = (TCP)
使用以下命令创建oracle数据库资源和listnener资源
[root@fmpracdb pcsd]# pcs resource create arap2db oracle sid=arap2 home=/home/app/11.2/db_1 user=oracle shutdown_method=immediate
[root@fmpracdb pcsd]# pcs resource create arap2_lsn oralsnr sid=arap2 home=/home/app/11.2/db_1 user=oracle listener=LSNR_arap2
[root@fmpracdb pcsd]# pcs resource create cvsdb oracle sid=cvs home=/home/app/11.2/db_1 user=oracle shutdown_method=immediate
[root@fmpracdb pcsd]# pcs resource create cvs_lsn oralsnr sid=cvs home=/home/app/11.2/db_1 user=oracle listener=LSNR_cvs2
使用以下命令查看当前数据库资源和listener资源配置
[root@fmpracora dinxin]# pcs resource show cvs_lsn
Resource: cvs_lsn (class=ocf provider=heartbeat type=oralsnr)
Attributes: sid=cvs home=/home/app/11.2/db_1 user=oracle listener=LSNR_cvs
Operations: start interval=0s timeout=120 (cvs_lsn-start-interval-0s)
stop interval=0s timeout=120 (cvs_lsn-stop-interval-0s)
monitor interval=10 timeout=30 (cvs_lsn-monitor-interval-10)
[root@fmpracora dinxin]# pcs resource show cvsdb
Resource: cvsdb (class=ocf provider=heartbeat type=oracle)
Attributes: sid=cvs home=/home/app/11.2/db_1 user=oracle shutdown_method=immediate
Operations: start interval=0s timeout=120 (cvsdb-start-interval-0s)
stop interval=0s timeout=120 (cvsdb-stop-interval-0s)
monitor interval=120 timeout=30 (cvsdb-monitor-interval-120)
[root@fmpracora dinxin]# pcs resource show arap2db
Resource: arap2db (class=ocf provider=heartbeat type=oracle)
Attributes: sid=arap2 home=/home/app/11.2/db_1 user=oracle shutdown_method=immediate
Operations: start interval=0s timeout=120 (arap2db-start-interval-0s)
stop interval=0s timeout=120 (arap2db-stop-interval-0s)
monitor interval=120 timeout=30 (arap2db-monitor-interval-120)
[root@fmpracora dinxin]# pcs resource show arap2_lsn
Resource: arap2_lsn (class=ocf provider=heartbeat type=oralsnr)
Attributes: sid=arap2 home=/home/app/11.2/db_1 user=oracle listener=LSNR_arap2
Operations: start interval=0s timeout=120 (arap2_lsn-start-interval-0s)
stop interval=0s timeout=120 (arap2_lsn-stop-interval-0s)
monitor interval=10 timeout=30 (arap2_lsn-monitor-interval-10)
[root@fmpracora dinxin]#
2.3.11 创建集群组
创建集群组,在组中添加浮动IP地址资源,文件系统资源,oracle listener资源,以及Oracle数据库资源,并设置资源间的依赖关系。组设置成功后,将作为一个整体进行节点间的切换。
# pcs resource group add ac_group fmpracora-vip
#pcs resource group add ac_group ac_fs --after fmpracora-vip
#pcs resource group add ac_group cvs_lsn --after ac_fs
#pcs resource group add ac_group cvsdb --after cvs_lsn
2.3.12 创建集群资源优先启动节点
集群节点将在优先节点启动。当优先节点因故障关机后,自动切换到其它节点。在优先节点开机重新启动后,将跟随cman,pcs服务启动而自动回切到优先节点。
#pcs constraint location fc_fs prefers fmprfcora-priv
#pcs constraint location fmprfcora-vip prefers fmprfcora-priv
#pcs constraint location cvs_lsn prefers fmprfcora-priv
#pcs constraint location cvsdb prefers fmprfcora-priv
#pcs constraint location fc_group prefers fmprfcora-priv
#pcs constraint location ac_fs prefers fmpracora-priv
#pcs constraint location fmpracora-vip prefers fmpracora-priv
#pcs constraint location apar2_lsn prefers fmpracora-priv
#pcs constraint location apar2db prefers fmpracora-priv
#pcs constraint location ac_group prefers fmpracora-priv
查看优先节点:
[root@fmpracora dinxin]# pcs constraint
Location Constraints:
Resource: ac_group
Enabled on: fmpracora-priv (score:INFINITY)
Resource: fc_group
Enabled on: fmprfcora-priv (score:INFINITY)
Resource: fmpracora-vip
Enabled on: fmpracora-priv (score:INFINITY)
Resource: fmprfcora-vip
Enabled on: fmprfcora-priv (score:INFINITY)
3RHEL 6.8 & 7.2 HA维护手册
3.1移动资源组到其它节点
[root@fmdrjdeora ~]# pcs resource move dr_group
Warning: Creating location constraint cli-ban-dr_group-on-fmdrjdeora-priv with a score of -INFINITY for resource dr_group on node fmdrjdeora-priv.
This will prevent dr_group from running on fmdrjdeora-priv until the constraint is removed. This will be the case even if fmdrjdeora-priv is the last node in the cluster.
服务迁移到其它节点。
[root@fmdrjdeora ~]# pcs resource
Resource Group: dr_group
fmdrjdeora-vip(ocf::heartbeat:IPaddr2):Started fmprjdeora-priv
dr_fs(ocf::heartbeat:Filesystem):Started fmprjdeora-priv
jdedbdr_lsn(ocf::heartbeat:oralsnr):Started fmprjdeora-priv
jdedr_db(ocf::heartbeat:oracle):Started fmprjdeora-priv
Resource Group: pr_group
pr_fs(ocf::heartbeat:Filesystem):Started fmprjdeora-priv
fmprjdeora-vip(ocf::heartbeat:IPaddr2):Started fmprjdeora-priv
jdedb(ocf::heartbeat:oracle):Started fmprjdeora-priv
jde_lsn(ocf::heartbeat:oralsnr):Started fmprjdeora-priv
且生成一个在节点fmdrjdeora-priv禁用
[root@fmdrjdeora oracle]# pcs constraint
Location Constraints:
Resource: dr_fs
Enabled on: fmdrjdeora-priv (score:INFINITY)
Resource: dr_group
Enabled on: fmdrjdeora-priv (score:300)
Enabled on: fmprjdeora-priv (score:100)
Disabled on: fmdrjdeora-priv (score:-INFINITY) (role: Started)
3.2回切该资源回到原优先节点
需要清除限制,否则该资源不会fmdrjdeora-priv运行。
[root@fmprjdeora ~]# pcs resource clear dr_group
清除资源后,constraint恢复正常。
[root@fmprjdeora ~]# pcs constraint
Location Constraints:
Resource: dr_fs
Enabled on: fmdrjdeora-priv (score:INFINITY)
Resource: dr_group
Enabled on: fmdrjdeora-priv (score:300)
Enabled on: fmprjdeora-priv (score:100)
Resource: fmdrjdeora-vip
Enabled on: fmdrjdeora-priv (score:300)
Resource: fmprjdeora-vip
Enabled on: fmprjdeora-priv (score:INFINITY)
Resource: jde_lsn
Enabled on: fmprjdeora-priv (score:INFINITY)
Resource: jdedb
Enabled on: fmprjdeora-priv (score:INFINITY)
Resource: pr_fs
Enabled on: fmprjdeora-priv (score:INFINITY)
Resource: pr_group
Enabled on: fmprjdeora-priv (score:INFINITY)
Ordering Constraints:
Resource Sets:
set fmprjdeora-vip pr_fs jde_lsn jdedb
set fmdrjdeora-vip dr_fs
Colocation Constraints:
资源就会自动回切到原始服务器。
[root@fmdrjdeora ~]# pcs resource clear dr_group
[root@fmdrjdeora ~]# pcs resource
Resource Group: dr_group
fmdrjdeora-vip(ocf::heartbeat:IPaddr2):Started fmdrjdeora-priv
dr_fs(ocf::heartbeat:Filesystem):Stopped
jdedbdr_lsn(ocf::heartbeat:oralsnr):Stopped
jdedr_db(ocf::heartbeat:oracle): Stopped
Resource Group: pr_group
pr_fs(ocf::heartbeat:Filesystem):Started fmprjdeora-priv
fmprjdeora-vip(ocf::heartbeat:IPaddr2):Started fmprjdeora-priv
jdedb(ocf::heartbeat:oracle):Started fmprjdeora-priv
jde_lsn(ocf::heartbeat:oralsnr):Started fmprjdeora-priv
[root@fmdrjdeora ~]# pcs resource
Resource Group: dr_group
fmdrjdeora-vip(ocf::heartbeat:IPaddr2):Started fmdrjdeora-priv
dr_fs(ocf::heartbeat:Filesystem):Started fmdrjdeora-priv
jdedbdr_lsn(ocf::heartbeat:oralsnr):Started fmdrjdeora-priv
jdedr_db(ocf::heartbeat:oracle):Started fmdrjdeora-priv
Resource Group: pr_group
pr_fs(ocf::heartbeat:Filesystem): Started fmprjdeora-priv
fmprjdeora-vip(ocf::heartbeat:IPaddr2):Started fmprjdeora-priv
jdedb(ocf::heartbeat:oracle):Started fmprjdeora-priv
jde_lsn(ocf::heartbeat:oralsnr):Started fmprjdeora-priv
3.3查看资源运行状态
[root@fmpracora dinxin]# pcs resource
Resource Group: ac_group
fmpracora-vip(ocf::heartbeat:IPaddr2):Started fmpracora-priv
ac_fs(ocf::heartbeat:Filesystem):Started fmpracora-priv
cvs_lsn(ocf::heartbeat:oralsnr): Started fmpracora-priv
cvsdb(ocf::heartbeat:oracle):Started fmpracora-priv
Resource Group: fc_group
fmprfcora-vip(ocf::heartbeat:IPaddr2):Started fmprfcora-priv
fc_fs(ocf::heartbeat:Filesystem):Started fmprfcora-priv
arap2_lsn(ocf::heartbeat:oralsnr):Started fmprfcora-priv
arap2db(ocf::heartbeat:oracle):Started fmprfcora-priv
[root@fmpracora dinxin]#
3.4查看资源运行状态
[root@fmpracora dinxin]# pcs status
Cluster name: fmprfcac_cluster
Last updated: Tue Aug 23 15:29:07 2016 Last change: Tue Aug 23 15:24:26 2016 by root via cibadmin on fmpracora-priv
Stack: corosync
Current DC: fmpracora-priv (version 1.1.13-10.el7-44eb2dd) - partition with quorum
2 nodes and 10 resources configured
Online: [ fmpracora-priv fmprfcora-priv ]
Full list of resources:
imm-fmprfcora(stonith:fence_ipmilan):Started fmpracora-priv
imm-fmpracora(stonith:fence_ipmilan):Started fmprfcora-priv
Resource Group: ac_group
fmpracora-vip(ocf::heartbeat:IPaddr2):Started fmpracora-priv
ac_fs(ocf::heartbeat:Filesystem):Started fmpracora-priv
cvs_lsn(ocf::heartbeat:oralsnr):Started fmpracora-priv
cvsdb(ocf::heartbeat:oracle): Started fmpracora-priv
Resource Group: fc_group
fmprfcora-vip(ocf::heartbeat:IPaddr2):Started fmprfcora-priv
fc_fs(ocf::heartbeat:Filesystem):Started fmprfcora-priv
arap2_lsn(ocf::heartbeat:oralsnr):Started fmprfcora-priv
arap2db(ocf::heartbeat:oracle):Started fmprfcora-priv
Failed Actions:
* cvsdb_monitor_120000 on fmpracora-priv 'not running' (7): call=245, status=complete, exitreason='none',
last-rc-change='Mon Aug 22 14:05:39 2016', queued=0ms, exec=0ms
* cvs_lsn_monitor_10000 on fmpracora-priv 'not running' (7): call=243, status=complete, exitreason='none',
last-rc-change='Tue Aug 23 10:45:13 2016', queued=0ms, exec=0ms
PCSD Status:
fmprfcora-priv: Online
fmpracora-priv: Online
Daemon Status:
corosync: active/enabled
pacemaker: active/enabled
pcsd: active/enabled
3.5standby节点
设置节点为standby,该节点的所有集群资源将自动切换到集群中的其它节点
[root@fmprpoora ~]# pcs cluster standby fmprpoora-priv
取消standby的命令是unstandby,该节点的所有集群资源将自动回切到本节点。
[root@fmprpoora ~]# pcs cluster unstandby fmprpoora-priv
3.6清除资源错误计数计
[root@fmpracora dinxin]# pcs resource cleanup
Waiting for 1 replies from the CRMd. OK
3.7维护节点
节点处于维护状态时,该节点上的资源将不会自动启动。
3.7.1 RHEL 6.8维护节点
[root@fmpridcora oracle]#pcs node maintenance fmpridcora-priv
[root@fmpridcora oracle]# pcs status
Cluster name: fmpridc-clu
Last updated: Tue Aug 23 14:56:21 2016 Last change: Tue Aug 23 14:55:59 2016 by root via crm_attribute on fmpridcora-priv
Stack: cman
Current DC: fmprbiora-priv (version 1.1.14-8.el6-70404b0) - partition with quoru m
2 nodes and 6 resources configured
Node fmpridcora-priv: maintenance
Online: [ fmprbiora-priv ]
Full list of resources:
imm-fmpridcora (stonith:fence_ipmilan): Started fmprbiora-priv
imm-fmprbiora(stonith:fence_ipmilan):Started fmpridcora-priv (unmanag ed)
Resource Group: idc_group
fmpridcora-vip (ocf::heartbeat:IPaddr2): Started fmpridcora-priv (unmanaged)
idc_fs(ocf::heartbeat:Filesystem): Started fmpridcora-priv (unmanaged)
fmidc_lsn(ocf::heartbeat:oralsnr):Started fmpridcora-priv (unmanaged)
fmidcdb(ocf::heartbeat:oracle): Started fmpridcora-priv (unmanaged)
PCSD Status:
fmpridcora-priv: Online
fmprbiora-priv: Online
清除节点维护状态:
[root@fmpridcora oracle]# pcs node unmaintenance fmpridcora-priv
[root@fmpridcora oracle]# pcs status
Cluster name: fmpridc-clu
Last updated: Tue Aug 23 15:32:28 2016 Last change: Tue Aug 23 15:31:19 2016 by root via crm_attribute on fmpridcora-priv
Stack: cman
Current DC: fmprbiora-priv (version 1.1.14-8.el6-70404b0) - partition with quorum
2 nodes and 6 resources configured
Online: [ fmprbiora-priv fmpridcora-priv ]
Full list of resources:
imm-fmpridcora (stonith:fence_ipmilan): Started fmprbiora-priv
imm-fmprbiora(stonith:fence_ipmilan):Started fmpridcora-priv
Resource Group: idc_group
fmpridcora-vip(ocf::heartbeat:IPaddr2):Started fmpridcora-priv
idc_fs(ocf::heartbeat:Filesystem):Started fmpridcora-priv
fmidc_lsn(ocf::heartbeat:oralsnr):Started fmpridcora-priv
fmidcdb(ocf::heartbeat:oracle):Started fmpridcora-priv
PCSD Status:
fmpridcora-priv: Online
fmprbiora-priv: Online
3.7.2 RHEL 7.2
[root@fmpracora dinxin]# pcs property set maintenance-mode=true
[root@fmpracora dinxin]# pcs property show maintenance-mode
Cluster Properties:
maintenance-mode: true
[root@fmpracora dinxin]# pcs status
Cluster name: fmprfcac_cluster
Last updated: Tue Aug 23 15:11:46 2016 Last change: Tue Aug 23 15:09:15 2016 by root via cibadmin on fmpracora-priv
Stack: corosync
Current DC: fmpracora-priv (version 1.1.13-10.el7-44eb2dd) - partition with quorum
2 nodes and 10 resources configured
Online: [ fmpracora-priv fmprfcora-priv ]
Full list of resources:
imm-fmprfcora(stonith:fence_ipmilan):Started fmpracora-priv (unmanaged)
imm-fmpracora(stonith:fence_ipmilan):Started fmprfcora-priv (unmanaged)
Resource Group: ac_group
fmpracora-vip(ocf::heartbeat:IPaddr2):Started fmpracora-priv (unmanaged)
ac_fs(ocf::heartbeat:Filesystem):Started fmpracora-priv (unmanaged)
cvs_lsn(ocf::heartbeat:oralsnr):Stopped (unmanaged)
cvsdb (ocf::heartbeat:oracle): Started fmpracora-priv (unmanaged)
Resource Group: fc_group
fmprfcora-vip(ocf::heartbeat:IPaddr2):Started fmprfcora-priv (unmanaged)
fc_fs(ocf::heartbeat:Filesystem):Started fmprfcora-priv (unmanaged)
arap2_lsn(ocf::heartbeat:oralsnr):Started fmprfcora-priv (unmanaged)
arap2db(ocf::heartbeat:oracle):Started fmprfcora-priv (unmanaged)
PCSD Status:
fmprfcora-priv: Online
fmpracora-priv: Online
Daemon Status:
corosync: active/enabled
pacemaker: active/enabled
pcsd: active/enabled
清除维护模式
[root@fmpracora dinxin]# pcs property set maintenance-mode=false
清除维护模式后,系统状态恢复正常。
[root@fmpracora dinxin]# pcs resource
Resource Group: ac_group
fmpracora-vip (ocf::heartbeat:IPaddr2): Started fmpracora-priv
ac_fs(ocf::heartbeat:Filesystem):Started fmpracora-priv
cvs_lsn(ocf::heartbeat:oralsnr):Started fmpracora-priv
cvsdb(ocf::heartbeat:oracle):Started fmpracora-priv
Resource Group: fc_group
fmprfcora-vip(ocf::heartbeat:IPaddr2):Started fmprfcora-priv
fc_fs(ocf::heartbeat:Filesystem):Started fmprfcora-priv
arap2_lsn(ocf::heartbeat:oralsnr):Started fmprfcora-priv
arap2db(ocf::heartbeat:oracle):Started fmprfcora-priv
3.8手工启动资源
当系统处于维护模式时,可以手工启动资源。启动资源加上--full参数,可以看到详细的输出。
[root@fmprpoora ~]# pcs resource debug-start fmprpoora-vip
Operation start for fmprpoora-vip (ocf:heartbeat:IPaddr2) returned 0
>stderr: INFO: Adding inet address 10.0.101.7/24 with broadcast address 10.0.101.255 to device bond0
>stderr: INFO: Bringing device bond0 up
>stderr: INFO: /usr/libexec/heartbeat/send_arp -i 200 -r 5 -p /var/run/resource-agents/send_arp-10.0.101.7 bond0 10.0.101.7 auto not_used not_used
3.9手工停止资源
当系统处于维护模式时,可以停止资源。
[root@fmprmaora ~]# pcs resource debug-stop po_fs
Operation stop for po_fs (ocf:heartbeat:Filesystem) returned 0
>stderr: INFO: Running stop for /dev/mapper/povg-polv on /po-oradata
>stderr: INFO: Trying to unmount /po-oradata
>stderr: INFO: unmounted /po-oradata successfully
[root@fmprmaora ~]# pcs resource debug-stop fmprpoora-vip
Operation stop for fmprpoora-vip (ocf:heartbeat:IPaddr2) returned 0
>stderr: INFO: IP status = ok, IP_CIP=
[root@fmprmaora ~]#
3.10删除cluster配置
[root@fmprpoora ~]# pcs cluster destroy
Shutting down pacemaker/corosync services...
Pacemaker Cluster Manager is already stopped [确定]
Stopping cluster:
Leaving fence domain... [确定]
Stopping gfs_controld... [确定]
Stopping dlm_controld... [确定]
Stopping fenced... [确定]
Stopping cman... [确定]
Unloading kernel modules... [确定]
Unmounting configfs... [确定]
Killing any remaining services...
Removing all cluster configuration files...
[root@fmprpoora ~]#
pcs resource disable<resource-group-name>