环境：RHEL 6.4 + Oracle 11.2.0.3 + ASM单实例
1.重启主机后,+DATA diskgroup启动不成功,现象如下：
[grid@JY-DB ~]$ crsctl stat res -t -------------------------------------------------------------------------------- NAME TARGET STATE SERVER STATE_DETAILS -------------------------------------------------------------------------------- Local Resources -------------------------------------------------------------------------------- ora.DATA.dg ONLINE OFFLINE jy-db ora.LISTENER.lsnr ONLINE ONLINE jy-db ora.asm ONLINE ONLINE jy-db Started ora.ons OFFLINE OFFLINE jy-db -------------------------------------------------------------------------------- Cluster Resources -------------------------------------------------------------------------------- ora.cssd 1 ONLINE ONLINE jy-db ora.diskmon 1 OFFLINE OFFLINE ora.evmd 1 ONLINE ONLINE jy-db ora.jyzhao.db 1 ONLINE OFFLINE Instance Shutdown
2.尝试手动启动data磁盘组失败，报错如下：
[grid@JY-DB ~]$ srvctl start diskgroup -g data PRCR-1079 : 无法启动资源 ora.DATA.dg CRS-5017: The resource action "ora.DATA.dg start" encountered the following error: ORA-15032: not all alterations performed ORA-15017: diskgroup "DATA" cannot be mounted ORA-15063: ASM discovered an insufficient number of disks for diskgroup "DATA" . For details refer to "(:CLSN00107:)" in "/u01/app/11.2.0/grid/log/jy-db/agent/ohasd/oraagent_grid/oraagent_grid.log". CRS-2674: Start of 'ora.DATA.dg' on 'jy-db' failed [grid@JY-DB ~]$ id uid=501(grid) gid=500(oinstall) 组=500(oinstall),501(dba),502(asmadmin),503(asmdba),504(asmoper) 环境=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 [grid@JY-DB ~]$
3.网上搜索得知Linux重启会重置raw设备的ownership.
On Linux, at least, a restart of the system will reset the ownership of raw devices. The usual trick is to have some config or init routine reset the ownership.
4.检查data磁盘组对应的磁盘设备属主，更改为grid:oinstall
[root@JY-DB ~]# cd /dev/mapper/ [root@JY-DB mapper]# ls control ora_vg-lv_asm1 ora_vg-lv_asm2 ora_vg-lv_asm3 ora_vg-lv_asm4 ora_vg-lv_usr2 VolGroup-lv_home VolGroup-lv_root VolGroup-lv_swap [root@JY-DB mapper]# ls -lh 总用量 0 crw-rw----. 1 root root 10, 58 10月 9 10:46 control lrwxrwxrwx. 1 root root 7 10月 9 10:46 ora_vg-lv_asm1 -> ../dm-2 lrwxrwxrwx. 1 root root 7 10月 9 10:46 ora_vg-lv_asm2 -> ../dm-3 lrwxrwxrwx. 1 root root 7 10月 9 10:46 ora_vg-lv_asm3 -> ../dm-4 lrwxrwxrwx. 1 root root 7 10月 9 10:46 ora_vg-lv_asm4 -> ../dm-5 lrwxrwxrwx. 1 root root 7 10月 9 10:46 ora_vg-lv_usr2 -> ../dm-6 lrwxrwxrwx. 1 root root 7 10月 9 10:46 VolGroup-lv_home -> ../dm-7 lrwxrwxrwx. 1 root root 7 10月 9 10:46 VolGroup-lv_root -> ../dm-0 lrwxrwxrwx. 1 root root 7 10月 9 10:46 VolGroup-lv_swap -> ../dm-1 [root@JY-DB mapper]# cd .. [root@JY-DB dev]# ls -lh 总用量 0 drwxr-xr-x. 2 root root 820 10月 9 10:46 block drwxr-xr-x. 2 root root 2.5K 10月 9 10:47 char crw-------. 1 root root 5, 1 10月 9 10:46 console lrwxrwxrwx. 1 root root 11 10月 9 10:46 core -> /proc/kcore drwxr-xr-x. 6 root root 120 10月 9 10:46 cpu crw-rw----. 1 root root 10, 61 10月 9 10:46 cpu_dma_latency crw-rw----. 1 root root 10, 62 10月 9 10:46 crash drwxr-xr-x. 5 root root 100 10月 9 10:46 disk brw-rw----. 1 root disk 253, 0 10月 9 10:46 dm-0 brw-rw----. 1 root disk 253, 1 10月 9 10:46 dm-1 brw-rw----. 1 root disk 253, 2 10月 9 10:46 dm-2 brw-rw----. 1 root disk 253, 3 10月 9 10:46 dm-3 brw-rw----. 1 root disk 253, 4 10月 9 10:46 dm-4 brw-rw----. 1 root disk 253, 5 10月 9 10:46 dm-5 brw-rw----. 1 root disk 253, 6 10月 9 10:46 dm-6 brw-rw----. 1 root disk 253, 7 10月 9 10:46 dm-7 …… crw-rw-rw-. 1 root root 1, 5 10月 9 10:46 zero [root@JY-DB dev]# chown grid:oinstall dm* [root@JY-DB dev]# ls -lh 总用量 0 drwxr-xr-x. 2 root root 820 10月 9 10:46 block drwxr-xr-x. 2 root root 2.5K 10月 9 10:47 char crw-------. 1 root root 5, 1 10月 9 10:46 console lrwxrwxrwx. 1 root root 11 10月 9 10:46 core -> /proc/kcore drwxr-xr-x. 6 root root 120 10月 9 10:46 cpu crw-rw----. 1 root root 10, 61 10月 9 10:46 cpu_dma_latency crw-rw----. 1 root root 10, 62 10月 9 10:46 crash drwxr-xr-x. 5 root root 100 10月 9 10:46 disk brw-rw----. 1 grid oinstall 253, 0 10月 9 10:46 dm-0 brw-rw----. 1 grid oinstall 253, 1 10月 9 10:46 dm-1 brw-rw----. 1 grid oinstall 253, 2 10月 9 10:46 dm-2 brw-rw----. 1 grid oinstall 253, 3 10月 9 10:46 dm-3 brw-rw----. 1 grid oinstall 253, 4 10月 9 10:46 dm-4 brw-rw----. 1 grid oinstall 253, 5 10月 9 10:46 dm-5 brw-rw----. 1 grid oinstall 253, 6 10月 9 10:46 dm-6 brw-rw----. 1 grid oinstall 253, 7 10月 9 10:46 dm-7 …… crw-rw-rw-. 1 root root 1, 5 10月 9 10:46 zero
5.此时再次尝试手动启动磁盘组成功
[grid@JY-DB ~]$ srvctl start diskgroup -g data [grid@JY-DB ~]$ crsctl stat res -t -------------------------------------------------------------------------------- NAME TARGET STATE SERVER STATE_DETAILS -------------------------------------------------------------------------------- Local Resources -------------------------------------------------------------------------------- ora.DATA.dg ONLINE ONLINE jy-db ora.LISTENER.lsnr ONLINE ONLINE jy-db ora.asm ONLINE ONLINE jy-db Started ora.ons OFFLINE OFFLINE jy-db -------------------------------------------------------------------------------- Cluster Resources -------------------------------------------------------------------------------- ora.cssd 1 ONLINE ONLINE jy-db ora.diskmon 1 OFFLINE OFFLINE ora.evmd 1 ONLINE ONLINE jy-db ora.jyzhao.db 1 ONLINE ONLINE jy-db Open [grid@JY-DB ~]$