问题现象
在使用shutdown abort停DataGuard备库后。备库不能open，报ORA-01196错误。
详细例如以下：
发现一备库不能应用日志。查看备库日志没发现报错。怀疑是备库应用日志服务停止，于是尝试重新启动备库。
可能由于备库是读业务比較繁忙，在shutdown immediate关闭备库时等时间过长，于是使用了shutdown abort命令；
但后面在启动备库时发生报错，造成数据文件损坏，控制文件和数据文件的scn号不一致。
--启动备库时报错
SQL> startup
ORACLE 例程已经启动。
Total System Global Area 2.0310E+10 bytes
Fixed Size 2235256 bytes
Variable Size 9328133256 bytes
Database Buffers 1.0939E+10 bytes
Redo Buffers 40894464 bytes
数据库装载完成。
ORA-10458: standby database requiresrecovery
ORA-01196: 文件 1 因为介质恢复会话失败而不一致
ORA-01110: 数据文件 1:'+DATA/htdb5/datafile/system.261.759082693'
--查看日志
alter database open
Data Guard Brokerinitializing...
Data Guard Brokerinitialization complete
Beginning standby crash recovery.
Serial Media Recovery started
Managed Standby Recoverystarting Real Time Apply
Media Recovery Log+FRA/htdb5/archivelog/2015_07_16/thread_1_seq_180068.1541.885192077
Thu Jul 16 12:00:47 2015
Errors in file/u01/app/ora11g/diag/rdbms/htdb5/htdb5/trace/htdb5_ora_10154.trc:
ORA-01013: 用户请求取消当前的操作
ORA-10567: Redo is inconsistentwith data block (file# 47, block# 1187724, file offset is 1139900416 bytes)
ORA-10564: tablespace JDYWP_IDX
ORA-01110: 数据文件 47:'+DATA/htdb5/datafile/jdywp_idx.336.856967805'
ORA-10561: block type'TRANSACTION MANAGED INDEX BLOCK', data object# 251837
Errors in file/u01/app/ora11g/diag/rdbms/htdb5/htdb5/trace/htdb5_ora_10154.trc:
ORA-00339: 归档日志未包括不论什么重做
ORA-00334: 归档日志: '+DATA/htdb5/onlinelog/group_2.280.759082845'
ORA-10567: Redo is inconsistentwith data block (file# 47, block# 1187724, file offset is 1139900416 bytes)
ORA-10564: tablespace JDYWP_IDX
ORA-01110: 数据文件 47:'+DATA/htdb5/datafile/jdywp_idx.336.856967805'
ORA-10561: block type'TRANSACTION MANAGED INDEX BLOCK', data object# 251837
Errors in file/u01/app/ora11g/diag/rdbms/htdb5/htdb5/trace/htdb5_ora_10154.trc (incident=116743):
ORA-00600: 内部错误代码, 參数: [3020],[47], [1187724], [198320012], [], [], [], [], [], [], [], []
ORA-10567: Redo is inconsistentwith data block (file# 47, block# 1187724, file offset is 1139900416 bytes)
ORA-10564: tablespace JDYWP_IDX
ORA-01110: 数据文件 47:'+DATA/htdb5/datafile/jdywp_idx.336.856967805'
ORA-10561: block type'TRANSACTION MANAGED INDEX BLOCK', data object# 251837
Incident details in:/u01/app/ora11g/diag/rdbms/htdb5/htdb5/incident/incdir_116743/htdb5_ora_10154_i116743.trc
Use ADRCI or Support Workbenchto package the incident.
See Note 411.1 at My OracleSupport for error and packaging details.
Standby crash recovery aborteddue to error 600.
Errors in file/u01/app/ora11g/diag/rdbms/htdb5/htdb5/trace/htdb5_ora_10154.trc:
ORA-00600: 内部错误代码, 參数: [3020],[47], [1187724], [198320012], [], [], [], [], [], [], [], []
ORA-10567: Redo is inconsistentwith data block (file# 47, block# 1187724, file offset is 1139900416 bytes)
ORA-10564: tablespace JDYWP_IDX
ORA-01110: 数据文件 47:'+DATA/htdb5/datafile/jdywp_idx.336.856967805'
ORA-10561: block type'TRANSACTION MANAGED INDEX BLOCK', data object# 251837
Recovery interrupted!
Some recovered datafiles maybeleft media fuzzy
Media recovery may continue butopen resetlogs may fail
Completed standby crashrecovery.
Errors in file/u01/app/ora11g/diag/rdbms/htdb5/htdb5/trace/htdb5_ora_10154.trc:
ORA-10458: standby databaserequires recovery
ORA-01196: 文件 1 因为介质恢复会话失败而不一致
ORA-01110: 数据文件 1:'+DATA/htdb5/datafile/system.261.759082693'
ORA-10458 signalled during:alter database open...
Thu Jul 16 12:00:49 2015
Sweep [inc][116743]: completed
Sweep [inc2][116743]: completed
Thu Jul 16 12:00:49 2015
Dumping diagnostic data indirectory=[cdmp_20150716120049], requested by (instance=1, osid=10154),summary=[incident=116743].
Thu Jul 16 12:01:50 2015
解决的方法：
把备库闪回到正常的状态的时点。
--前提数据库闪回之前已经打开
SQL> select FLASHBACK_ON from v$database;
FLASHBACK_ON
------------------
YES
SQL> Flashback database to timestamp to_timestamp('2015-07-16 4:00:05','yyyy-mm-ddhh24:mi:ss');
--或是使用Flashbackdatabase to scn 947921
SQL> alter database open;
SQL> select open_mode from v$database;
OPEN_MODE
--------------------
READ ONLY
--启动实时应用
SQL> ALTER DATABASE RECOVER MANAGED STANDBY DATABASE USING CURRENT LOGFILE DISCONNECT;
SQL> select open_mode from v$database;
OPEN_MODE
--------------------
READ ONLY WITH APPLY
--查看日志看到日志已经从闪回的时点開始应用
Thu Jul 16 13:36:01 2015
Flashback database to timestampto_timestamp('2015-07-16 4:00:05','yyyy-mm-dd hh24:mi:ss')
Flashback Restore Start
Thu Jul 16 13:39:30 2015
Flashback Restore Complete
Flashback Media Recovery Start
started logmerger process
Parallel Media Recovery startedwith 16 slaves
Flashback Media Recovery Log+FRA/htdb5/archivelog/2015_07_16/thread_1_seq_180047.2212.885180637
Thu Jul 16 13:41:54 2015
Flashback Media Recovery Log+FRA/htdb5/archivelog/2015_07_16/thread_1_seq_180061.2611.885182343
Thu Jul 16 13:42:04 2015
Flashback Media Recovery Log+FRA/htdb5/archivelog/2015_07_16/thread_1_seq_180062.2861.885182537
Thu Jul 16 13:42:12 2015
Incomplete Recovery applieduntil change 71489772016 time 07/16/2015 04:00:06
Flashback Media RecoveryComplete
Completed: Flashback databaseto timestamp to_timestamp('2015-07-16 4:00:05','yyyy-mm-dd hh24:mi:ss')
Thu Jul 16 13:43:25 2015
Deleted Oracle managed file+FRA/htdb5/archivelog/2015_07_15/thread_1_seq_179690.2885.885083087
Thu Jul 16 13:43:25 2015
Standby controlfile consistentwith primary
RFS[3]: Selected log 8 forthread 1 sequence 180122 dbid 1083719948 branch 759079182
Archived Log entry 180115 addedfor thread 1 sequence 180121 ID 0x40a48484 dest 1:
Thu Jul 16 13:45:41 2015
alter database open
Data Guard Brokerinitializing...
Data Guard Brokerinitialization complete
SMON: enabling cache recovery
Dictionary check beginning
Dictionary check complete
Database Characterset isZHS16GBK
No Resource Manager plan active
replication_dependency_trackingturned off (no async multimaster replication found)
Physical standby databaseopened for read only access.
Completed: alter database open
Thu Jul 16 13:45:44 2015
ALTER DATABASE RECOVER MANAGEDSTANDBY DATABASE THROUGH ALL SWITCHOVERDISCONNECT USING CURRENT LOGFILE
Attempt to start backgroundManaged Standby Recovery process (htdb5)
Thu Jul 16 13:45:44 2015
MRP0 started with pid=51, OSid=14743
MRP0: Background ManagedStandby Recovery process started (htdb5)
started logmerger process
Thu Jul 16 13:45:50 2015
Managed Standby Recoverystarting Real Time Apply
Parallel Media Recovery startedwith 16 slaves
Waiting for all non-currentORLs to be archived...
All non-current ORLs have beenarchived.
Media Recovery Log +FRA/htdb5/archivelog/2015_07_16/thread_1_seq_180062.2861.885182537
Completed: ALTER DATABASERECOVER MANAGED STANDBY DATABASE THROUGHALL SWITCHOVER DISCONNECT USING CURRENTLOGFILE
Thu Jul 16 13:46:08 2015
Media Recovery Log+FRA/htdb5/archivelog/2015_07_16/thread_1_seq_180063.3683.885182777
Thu Jul 16 13:46:35 2015
Media Recovery Log+FRA/htdb5/archivelog/2015_07_16/thread_1_seq_180064.2542.885183119
Thu Jul 16 13:47:07 2015
Media Recovery Log+FRA/htdb5/archivelog/2015_07_16/thread_1_seq_180065.2717.885183615