VCS 一次使用。

One database was down today. So I login the server try to check and fix it. I found the Oracle home do not exist. So I check the /etc/fstab and found below.

[root@aaaa~]# cat /etc/fstab
# This file is edited by fstab-sync - see 'man fstab-sync' for details
LABEL=/                 /                       ext3    defaults        1 1
LABEL=/boot             /boot                   ext3    defaults        1 2
none                    /dev/pts                devpts  gid=5,mode=620  0 0
none                    /dev/shm                tmpfs   defaults        0 0
none                    /proc                   proc    defaults        0 0
none                    /sys                    sysfs   defaults        0 0
LABEL=SW-cciss/c0d0p3   swap                    swap    defaults        0 0
/dev/vx/dsk/orabin1adg/orabin   /opt/oracle/product/10.2        vxfs    defaults 1 2

The file system /dev/vx/dsk/orabin1adg/orabin is the directory should be mounted but not. It is controlled by VCS so I try to mount it with VCS.

First I use hastatus to check.  Hit below

[root@aaaa~]# hastatus
attempting to connect....
VCS ERROR V-16-1-10600 Cannot connect to VCS engine
attempting to connect....not available; will retry
attempting to connect....retrying
VCS ERROR V-16-1-10600 Cannot connect to VCS engine
attempting to connect....not available; will retry
attempting to connect....retrying
VCS ERROR V-16-1-10600 Cannot connect to VCS engine
attempting to connect....not available; will retry

Then I start the  VCS

[root@pss1sitdb1a ~]# /etc/init.d/vcs start
Starting vcs:                                              [  OK  ]

Then I use hastatus to check

[root@pss1sitdb1a ~]# hastatus
attempting to connect....
attempting to connect....connected


group           resource             system               message
--------------- -------------------- -------------------- --------------------
                                     aaaa          RUNNING
MultiNIC_sg                          aaaa          ONLINE
cvm                                  aaaa          STARTING OFFLINE
nfs_sg                               aaaa          OFFLINE
-------------------------------------------------------------------------

Try to online the nfs_sg hit below

[root@aaaaa~]# hagrp -online nfs_sg -sys aaaa
VCS WARNING V-16-1-10154 Group nfs_sg is frozen

Then

[root@aaaaa~]# hagrp -unfreeze nfs_sg -persistent
VCS WARNING V-16-1-11309 Configuration must be ReadWrite

Here is a quota which can explain the frozen.

Freezing a Service Group
凝固一个服务组是为了防止它离线；上线；或者failover.
当你凝固一个服务组时，VCS继续监视这个资源，但是不允许服务组(或者它的资源)背离线或者上线，failover也被禁止，即使资源故障。
注意：当凝固的时，VCS不能在服务组上面采取行动即使你通过带起服务上线在另一个系统上导致并行扰乱。
命令：
hagrp -freeze service_group
hagrp -unfreeze service_group
为了永久凝固一个服务组，你必须先要打开配置
haconf -makerw
hagrp -freeze service_group -persistent
hagrp -unfreeze service_group -persistent
为了确定是否一个服务组被凝固显示Frozen(永久性)和TFrozen(零时性)服务组属性对于一个服务组
hagrp -display service_group -attribute Frozen

Ok Then I 

[root@aaaa~]# haconf -makerw
[root@aaaa ~]# hagrp -unfreeze nfs_sg -persistent
[root@aaaa~]# hagrp -online nfs_sg -sys aaaa
VCS WARNING V-16-1-10159 Group nfs_sg is auto-disabled in cluster. This can happen if group is not probed on all alive nodes in group's SystemList or VCS engine is not running on all alive nodes in group's SystemList

Then I found. FUCK. I was checking the wrong enviroment. Be fucking careful next time please

Good thing is that I got some understanding about the VCS.