今天在给ASM扩磁盘组的时候发现11.2.0.2上asmcmd中lsdg命令所显示的磁盘组数不正确，现象如下:
Node A lsdg正常:
ASMCMD> lsdg
State Type Rebal Sector Block AU Total_MB Free_MB Req_mir_free_MB Usable_file_MB Offline_disks Voting_files Name
MOUNTED EXTERN N 512 4096 1048576 142976 50596 0 50596 0 Y DATA/
MOUNTED EXTERN N 512 4096 1048576 51200 50742 0 50742 0 N FRA/
Node B lsdg仅显示DATA磁盘组
ASMCMD> lsdg
State Type Rebal Sector Block AU Total_MB Free_MB Req_mir_free_MB Usable_file_MB Offline_disks Voting_files Name
MOUNTED EXTERN N 512 4096 1048576 142976 50596 0 50596 0 Y DATA/
从Node B节点上ASM实例的动态视图查询显示正常:
SQL> select name from v$asm_diskgroup;
NAME
------------------------------
DATA
FRA
ASM diag info:
SQL> select sysdate "Date and Time" from dual;
Date and Time
--------------------
28-JUN-2011 23:30:27
SQL>
SQL> select * from v$asm_diskgroup order by 1;
select * from v$asm_disk order by 1, 2, 3;
select * from gv$asm_operation order by 1;
select * from v$version where banner like '%Database%' order by 1;
select * from gv$asm_client order by 1;
prompt
prompt ASM Disk Groups
prompt ===============
select group_number "Group"
, name "Group Name"
, state "State"
, type "Type"
, total_mb/1024 "Total GB"
, free_mb/1024 "Free GB"
from v$asm_diskgroup
/
prompt
prompt ASM Disks
prompt ==============
col "Group" form 999
col "Disk" form 999
col "Header" form a9
col "Mode" form a8
GROUP_NUMBER NAME SECTOR_SIZE BLOCK_SIZE ALLOCATION_UNIT_SIZE STATE TYPE TOTAL_MB FREE_MB HOT_USED_MB COLD_USED_MB REQUIRED_MIRROR_FREE_MB USABLE_FILE_MB OFFLINE_DISKS COMPATIBILITY DATABASE_COMPATIBILITY V
------------ ------------------------------ ----------- ---------- -------------------- --------------- ------- ---------- ---------- ----------- ------------ ----------------------- -------------- ------------- ------------------------------------------------------------ ------------------------------------------------------------ -
0 FRA 0 4096 0 DISMOUNTED 0 0 0 0 0 0 0 0.0.0.0.0 0.0.0.0.0 N
1 DATA 512 4096 1048576 MOUNTED EXTERN 142976 50596 0 92380 0 50596 0 11.2.0.0.0 10.1.0.0.0 Y
SQL> col "Redundancy" form a10
col "Failure Group" form a10
col "Path" form a19
select group_number "Group"
, disk_number "Disk"
, header_status "Header"
GROUP_NUMBER DISK_NUMBER COMPOUND_INDEX INCARNATION MOUNT_S HEADER_STATU MODE_ST STATE REDUNDA LIBRARY OS_MB TOTAL_MB FREE_MB HOT_USED_MB COLD_USED_MB NAME FAILGROUP LABEL
------------ ----------- -------------- ----------- ------- ------------ ------- --------------- ------- ---------------------------------------------------------------- ---------- ---------- ---------- ----------- ------------ ------------------------------ ------------------------------ -------------------------------
PATH UDID PRODUCT CREATE_DATE MOUNT_DATE REPAIR_TIMER READS WRITES READ_ERRS WRITE_ERRS READ_TIME WRITE_TIME BYTES_READ BYTES_WRITTEN
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- ---------------------------------------------------------------- -------------------------------- -------------------- -------------------- ------------ ---------- ---------- ---------- ---------- ---------- ---------- ---------- -------------
P HASH_VALUE HOT_READS HOT_WRITES HOT_BYTES_READ HOT_BYTES_WRITTEN COLD_READS COLD_WRITES COLD_BYTES_READ COLD_BYTES_WRITTEN V SECTOR_SIZE FAILGRO
- ---------- ---------- ---------- -------------- ----------------- ---------- ----------- --------------- ------------------ - ----------- -------
0 0 0 3915932260 CLOSED MEMBER ONLINE NORMAL UNKNOWN System 51200 0 0 0 0
/dev/raw/raw8 28-JUN-2011 17:08:16 28-JUN-2011 17:08:25 0
0 N 512 REGULAR
1 0 16777216 3915932274 CACHED MEMBER ONLINE NORMAL UNKNOWN System 8096 8096 2827 0 5269 DATA_0000 DATA_0000
/dev/raw/raw1 29-MAR-2011 22:39:53 31-MAY-2011 06:42:41 0 26 1 0 0 .948212 .001678 1118208 4096
1452020686 0 0 0 0 0 0 0 0 Y 512 REGULAR
1 1 16777217 3915932273 CACHED MEMBER ONLINE NORMAL UNKNOWN System 8096 8096 2861 0 5235 DATA_0001 DATA_0001
/dev/raw/raw2 29-MAR-2011 22:39:53 31-MAY-2011 06:42:41 0 15 0 0 0 .868808 0 1105920 0
559735068 0 0 0 0 0 0 0 0 N 512 REGULAR
1 2 16777218 3915932272 CACHED MEMBER ONLINE NORMAL UNKNOWN System 8096 8096 2859 0 5237 DATA_0002 DATA_0002
/dev/raw/raw3 29-MAR-2011 23:36:28 31-MAY-2011 06:42:41 0 14 0 0 0 .738089 0 57344 0
2179807744 0 0 0 0 0 0 0 0 N 512 REGULAR
1 4 16777220 3915932270 CACHED MEMBER ONLINE NORMAL UNKNOWN System 8192 8192 2896 0 5296 DATA_0004 DATA_0004
/dev/raw/raw5 21-APR-2011 00:57:45 31-MAY-2011 06:42:41 0 14 0 0 0 .738083 0 57344 0
3104258115 0 0 0 0 0 0 0 0 N 512 REGULAR
1 5 16777221 3915932271 CACHED MEMBER ONLINE NORMAL UNKNOWN System 8096 8096 2856 0 5240 DATA_0005 DATA_0005
/dev/raw/raw4 30-MAY-2011 20:36:59 31-MAY-2011 06:42:41 0 15 0 0 0 .911934 0 1105920 0
2075303794 0 0 0 0 0 0 0 0 N 512 REGULAR
1 7 16777223 3915932269 CACHED MEMBER ONLINE NORMAL UNKNOWN System 51200 51200 18149 0 33051 DATA_0007 DATA_0007
/dev/raw/raw6 28-JUN-2011 17:09:22 28-JUN-2011 17:09:22 0 39 5 0 0 .998594 6.022413 159744 20480
, mode_status "Mode"
2166785847 0 0 0 0 18 0 73728 0 N 512 REGULAR
1 8 16777224 3915932268 CACHED MEMBER ONLINE NORMAL UNKNOWN System 51200 51200 18148 0 33052 DATA_0008 DATA_0008
/dev/raw/raw7 28-JUN-2011 17:09:58 28-JUN-2011 17:09:58 0 25 0 0 0 .760427 0 102400 0
1385034587 0 0 0 0 6 0 24576 0 N 512 REGULAR
8 rows selected.
SQL> , state "State"
, redundancy "Redundancy"
, total_mb "Total MB"
, free_mb "Free MB"
, name "Disk Name"
, failgroup "Failure Group"
, path "Path"
from v$asm_disk
order by group_number
, disk_number
/
prompt
prompt Instances currently accessing these diskgroups
prompt ==============================================
no rows selected
SQL>
BANNER
--------------------------------------------------------------------------------
Oracle Database 11g Enterprise Edition Release 11.2.0.2.0 - 64bit Production
SQL>
select c.group_number "Group"
, g.name "Group Name"
, c.instance_name "Instance"
INST_ID GROUP_NUMBER INSTANCE_NAME DB_NAME STATUS SOFTWARE_VERSION COMPATIBLE_VERSION
---------- ------------ ---------------------------------------------------------------- -------- ------------ ------------------------------------------------------------ ------------------------------------------------------------
1 1 +ASM1 +ASM CONNECTED 11.2.0.2.0 11.2.0.2.0
1 2 PROD1 PROD CONNECTED 11.2.0.2.0 11.2.0.0.0
1 1 PROD1 PROD CONNECTED 11.2.0.2.0 11.2.0.0.0
2 1 +ASM2 +ASM CONNECTED 11.2.0.2.0 11.2.0.2.0
SQL> SQL>
SQL> SQL> ASM Disk Groups
SQL> ===============
SQL> SQL> 2 3 4 5 6 7 8 from v$asm_client c
, v$asm_diskgroup g
where g.group_number=c.group_number
/
Group Group Name State Type Total GB Free GB
---------- ------------------------- --------------- ------- ---------- -------
1 DATA MOUNTED EXTERN 139.625 49
0 FRA DISMOUNTED 0 0
SQL> SQL>
SQL> ASM Disks
SQL> ==============
SQL> SQL> SQL> SQL> SQL> SQL> SQL> SQL> SQL> SQL> 2 3 prompt
4 5 6 7 8 9 10 11 12 13 14 15 SQL> prompt Report the Percentage of Imbalance in all Mounted Diskgroups
prompt ==============================================
select dfail, count(dfail) from
(
select disk, count(failgroup) as dfail
from x$kfdpartner, v$asm_disk where
number_kfdpartner=disk_number and grp=group_number
Group Disk Header Mode State Redundancy Total MB Free MB Disk Name Failure Gr Path
----- ---- --------- -------- --------------- ---------- ---------- ---------- ------------------------------ ---------- -------------------
0 0 MEMBER ONLINE NORMAL UNKNOWN 0 0 /dev/raw/raw8
1 0 MEMBER ONLINE NORMAL UNKNOWN 8096 2827 DATA_0000 DATA_0000 /dev/raw/raw1
1 1 MEMBER ONLINE NORMAL UNKNOWN 8096 2861 DATA_0001 DATA_0001 /dev/raw/raw2
1 2 MEMBER ONLINE NORMAL UNKNOWN 8096 2859 DATA_0002 DATA_0002 /dev/raw/raw3
1 4 MEMBER ONLINE NORMAL UNKNOWN 8192 2896 DATA_0004 DATA_0004 /dev/raw/raw5
1 5 MEMBER ONLINE NORMAL UNKNOWN 8096 2856 DATA_0005 DATA_0005 /dev/raw/raw4
1 7 MEMBER ONLINE NORMAL UNKNOWN 51200 18149 DATA_0007 DATA_0007 /dev/raw/raw6
1 8 MEMBER ONLINE NORMAL UNKNOWN 51200 18148 DATA_0008 DATA_0008 /dev/raw/raw7
8 rows selected.
SQL> SQL>
SQL> Instances currently accessing these diskgroups
SQL> ==============================================
SQL> SQL> 2 3 4 5 6 7 group by disk, failgroup
)
group by dfail;
select g.name as "GROUP", d.name as "DISK", d.failgroup, fcnt, pcnt,
decode(pcnt - fcnt, 0, 'MUST', 'SHOULD') as action from
Group Group Name Instance
----- ------------------------- ----------------------------------------------------------------
1 DATA +ASM2
SQL> SQL>
SQL> Report the Percentage of Imbalance in all Mounted Diskgroups
SQL> ==============================================
SQL> 2 3 4 5 6 7 8 (select gnum, DISK1, failgroup, count(failgroup) as fcnt from
(select gnum, DISK1
from
(
select d.group_number as gnum, disk as disk1,
count(distinct failgroup) as dfail
from x$kfdpartner, v$asm_disk_stat d where
number_kfdpartner=disk_number and grp=d.group_number
no rows selected
SQL> SQL> 2 3 4 5 6 7 8 9 10 11 and active_kfdpartner=1
12 group by d.group_number, disk
13 ), v$asm_disk_stat
14 where dfail < 3
15 and disk1=disk_number
16 and gnum=group_number),
17 x$kfdpartner, v$asm_disk_stat d where
18 number_kfdpartner=disk_number and grp=d.group_number and grp=gnum
19 and disk1=disk
20 and active_kfdpartner=1
21 group by gnum, disk1, failgroup),
22 (select grp, disk, count(disk) as pcnt from x$kfdpartner where
23 active_kfdpartner=1 group by grp, disk),
24 v$asm_diskgroup_stat g, v$asm_disk_stat d
25 where gnum=grp and gnum=g.group_number and gnum=d.group_number and
26 disk=disk1 and disk=disk_number and
27 ((fcnt = 1 and (pcnt - fcnt) > 3) or ((pcnt - fcnt) = 0))
28 /
no rows selected
SQL>
SQL> col TYPE form a15
SQL> col FILE_NUMBER form 9999 head FILE_NUM
SQL> col GROUP_NUMBER form 9999 head GR_NUM
SQL> col GB for 9999.99
SQL>
SQL> select GROUP_NUMBER ,
2 FILE_NUMBER ,
3 COMPOUND_INDEX ,
4 INCARNATION ,
5 BLOCK_SIZE ,
6 BLOCKS ,
7 BYTES/1024/1024/1024 GB ,
8 TYPE ,
9 STRIPED ,
10 CREATION_DATE ,
11 MODIFICATION_DATE
12 from v$asm_file
13 where TYPE != 'ARCHIVELOG'
14 /
GR_NUM FILE_NUM COMPOUND_INDEX INCARNATION BLOCK_SIZE BLOCKS GB TYPE STRIPE CREATION_DATE MODIFICATION_DATE
------ -------- -------------- ----------- ---------- ---------- -------- --------------- ------ -------------------- --------------------
1 253 16777469 747096005 512 3 .00 ASMPARAMETERFIL COARSE 29-MAR-2011 22:40:05 29-MAR-2011 22:00:00
E
1 255 16777471 747096007 4096 66591 .25 OCRFILE COARSE 29-MAR-2011 22:40:07 28-JUN-2011 19:00:00
1 256 16777472 747100091 8192 116481 .89 DATAFILE COARSE 29-MAR-2011 23:48:10 28-JUN-2011 18:00:00
1 257 16777473 747100091 8192 122881 .94 DATAFILE COARSE 29-MAR-2011 23:48:11 28-JUN-2011 18:00:00
1 258 16777474 747100093 8192 78721 .60 DATAFILE COARSE 29-MAR-2011 23:48:12 28-JUN-2011 18:00:00
1 259 16777475 747100093 8192 883201 6.74 DATAFILE COARSE 29-MAR-2011 23:48:12 28-JUN-2011 18:00:00
1 260 16777476 747100215 16384 1545 .02 CONTROLFILE FINE 29-MAR-2011 23:50:14 28-JUN-2011 19:00:00
1 261 16777477 747100215 16384 1545 .02 CONTROLFILE FINE 29-MAR-2011 23:50:14 28-JUN-2011 19:00:00
1 262 16777478 747100217 512 102401 .05 ONLINELOG COARSE 29-MAR-2011 23:50:16 28-JUN-2011 22:00:00
1 263 16777479 747100219 512 102401 .05 ONLINELOG COARSE 29-MAR-2011 23:50:18 28-JUN-2011 22:00:00
1 264 16777480 747100219 512 102401 .05 ONLINELOG COARSE 29-MAR-2011 23:50:19 28-JUN-2011 20:00:00
1 265 16777481 747100223 512 102401 .05 ONLINELOG COARSE 29-MAR-2011 23:50:22 28-JUN-2011 20:00:00
1 266 16777482 747100227 8192 1889537 14.42 TEMPFILE COARSE 29-MAR-2011 23:50:27 28-JUN-2011 23:00:00
1 267 16777483 747100231 8192 44241 .34 DATAFILE COARSE 29-MAR-2011 23:50:31 28-JUN-2011 19:00:00
1 268 16777484 747100469 8192 144001 1.10 DATAFILE COARSE 29-MAR-2011 23:54:29 28-JUN-2011 18:00:00
prompt
1 269 16777485 747100547 512 102401 .05 ONLINELOG COARSE 29-MAR-2011 23:55:47 28-JUN-2011 19:00:00
1 270 16777486 747100549 512 102401 .05 ONLINELOG COARSE 29-MAR-2011 23:55:48 28-JUN-2011 19:00:00
1 271 16777487 747100549 512 102401 .05 ONLINELOG COARSE 29-MAR-2011 23:55:49 28-JUN-2011 19:00:00
1 272 16777488 747100553 512 102401 .05 ONLINELOG COARSE 29-MAR-2011 23:55:52 28-JUN-2011 19:00:00
1 273 16777489 747100553 512 9 .00 PARAMETERFILE COARSE 29-MAR-2011 23:55:53 12-MAY-2011 17:00:00
prompt free ASM disks and their paths
1 274 16777490 754587291 8192 2561 .02 DATAFILE COARSE 23-JUN-2011 15:34:50 28-JUN-2011 18:00:00
1 419 16777635 754281899 4096 91 .00 DUMPSET COARSE 20-JUN-2011 02:44:58 20-JUN-2011 02:00:00
1 657 16777873 755047049 8192 256001 1.95 TEMPFILE COARSE 28-JUN-2011 23:17:28 28-JUN-2011 23:00:00
1 658 16777874 755032881 8192 8171521 62.34 DATAFILE COARSE 28-JUN-2011 19:21:20 28-JUN-2011 22:00:00
24 rows selected.
SQL> SQL>
SQL> free ASM disks and their paths
SQL> prompt ===========================
===========================
SQL> select header_status , mode_status, path from V$asm_disk
2 where header_status in ('FORMER','CANDIDATE')
3 /
show parameter asm
show parameter size
show parameter proc
show parameter cluster
show parameter instance_type
show parameter instance_name
show parameter pfile
show sga
spool off
no rows selected
SQL> SQL>
NAME TYPE VALUE
------------------------------------ --------------- ------------------------------
asm_diskgroups string
asm_diskstring string /dev/raw/raw*
asm_power_limit integer 10
asm_preferred_read_failure_groups string
SQL>
NAME TYPE VALUE
------------------------------------ --------------- ------------------------------
db_cache_size big integer 0
large_pool_size big integer 12M
max_dump_file_size string unlimited
sga_max_size big integer 272M
shared_pool_reserved_size big integer 7969177
shared_pool_size big integer 0
sort_area_size integer 65536
workarea_size_policy string AUTO
SQL>
NAME TYPE VALUE
------------------------------------ --------------- ------------------------------
processes integer 100
SQL>
NAME TYPE VALUE
------------------------------------ --------------- ------------------------------
cluster_database boolean TRUE
cluster_database_instances integer 4
cluster_interconnects string
SQL>
NAME TYPE VALUE
------------------------------------ --------------- ------------------------------
instance_type string asm
SQL>
NAME TYPE VALUE
------------------------------------ --------------- ------------------------------
instance_name string +ASM2
SQL> SQL>
NAME TYPE VALUE
------------------------------------ --------------- ------------------------------
spfile string +DATA/rh-cluster/asmparameterf
ile/registry.253.747096005
SQL> SQL>
Total System Global Area 283930624 bytes
Fixed Size 2225792 bytes
Variable Size 256539008 bytes
ASM Cache 25165824 bytes
原来是因为lsdg只列出已经mount的diskgroup，因为在Node B上FRA磁盘组没有mount所以不被列出，我本来以为是Bug造成的。
感谢Liqin Zhang网友的指出。