主库添加数据文件后，备库因为创建文件失败而停止数据同步

primary环境：11gR2 RAC+ASM
standby环境：本地单机

主库添加数据文件后，备库因为创建文件失败而停止数据同步。

1.查看主库环境的数据文件

SQL> select file#,status,name from v$datafile;

     FILE# STATUS  NAME
---------- ------- --------------------------------------------
         1 SYSTEM  +ABCE/test/datafile/system.293.930567941
         2 ONLINE  +ABCE/test/datafile/sysaux.294.930567941
         3 ONLINE  +ABCE/test/datafile/undotbs1.323.937148625
         4 ONLINE  +ABCE/test/datafile/users.296.930567941
         5 ONLINE  +ABCE/test/datafile/undotbs2.324.937148779
         6 ONLINE  +ABCE/test/datafile/dtest.306.930571875
         7 ONLINE  +ABCE/test/datafile/dtest.328.950783567
         8 ONLINE  +ABCE/test/datafile/dtest.295.981369995  
         9 ONLINE  +ABCE/test/datafile/dtest.325.931996247
        10 ONLINE  +ABCE/test/datafile/dtest.326.931996261
        11 ONLINE  +ABCE/test/datafile/dtest.327.931996267

　

2.查看备库的数据文件

SQL> select file#,status,name from v$datafile;

     FILE# STATUS  NAME
---------- ------- ------------------------------------------------------------------
         1 SYSTEM  /t01/app/oracle/oradata/teststby/datafile/system.293.930567941
         2 ONLINE  /t01/app/oracle/oradata/teststby/datafile/sysaux.294.930567941
         3 ONLINE  /t01/app/oracle/oradata/teststby/datafile/undotbs1.323.937148625
         4 ONLINE  /t01/app/oracle/oradata/teststby/datafile/users.296.930567941
         5 ONLINE  /t01/app/oracle/oradata/teststby/datafile/undotbs2.324.937148779
         6 ONLINE  /t01/app/oracle/oradata/teststby/datafile/dtest.306.930571875
         7 ONLINE  /t01/app/oracle/oradata/teststby/datafile/dtest.328.950783567
         8 RECOVER /t01/app/oracle/testuct/11.2.0/dbhome_1/dbs/UNNAMED00008
         9 ONLINE  /t01/app/oracle/oradata/teststby/datafile/dtest.325.931996247
        10 ONLINE  /t01/app/oracle/oradata/teststby/datafile/dtest.326.931996261
        11 ONLINE  /t01/app/oracle/oradata/teststby/datafile/dtest.327.931996267

　　

3.查看备库的alert日志

Fri Jul 13 10:49:06 2018
Archived Log entry 10155 added for thread 2 sequence 5489 ID 0x1468c5ce dest 1:
RFS[8]: Selected log 11 for thread 2 sequence 5490 dbid 342406353 branch 930568019
Archived Log entry 10156 added for thread 2 sequence 5490 ID 0x1468c5ce dest 1:
RFS[8]: Selected log 11 for thread 2 sequence 5491 dbid 342406353 branch 930568019
Errors in file /t01/app/oracle/diag/rdbms/teststby/teststby/trace/teststby_pr00_3183.trc:
ORA-01119: error in creating database file '+ABCE'
ORA-17502: ksfdcre:4 Failed to create file +ABCE
ORA-15001: diskgroup "DATA" does not exist or is not mounted
ORA-15077: could not locate ASM instance serving a required diskgroup
ORA-29701: unable to connect to Cluster Synchronization Service
File #8 added to control file as 'UNNAMED00008'.
Originally created as:
'+ABCE/test/datafile/dtest.295.981369995'
Recovery was unable to create the file as a new OMF file.
Errors with log /t01/app/oracle/arch/2_5487_930568019.dbf
MRP0: Background Media Recovery terminated with error 1274
Errors in file /t01/app/oracle/diag/rdbms/teststby/teststby/trace/teststby_pr00_3183.trc:
ORA-01274: cannot add datafile '+ABCE/test/datafile/dtest.295.981369995' - file could not be created
Archived Log entry 10157 added for thread 1 sequence 7113 ID 0x1468c5ce dest 1:
Fri Jul 13 10:49:11 2018
RFS[7]: Selected log 9 for thread 1 sequence 7114 dbid 342406353 branch 930568019
Fri Jul 13 10:50:16 2018
Recovery interrupted!
Recovered data files to a consistent state at change 1162173286
Fri Jul 13 10:50:16 2018
MRP0: Background Media Recovery process shutdown (teststby)

　　

修复方案

在备库执行以下操作

SQL> show parameter STANDBY_FILE_MANAGEMENT

NAME                                 TYPE        VALUE
------------------------------------ ----------- ------------------------------
standby_file_management              string      AUTO
SQL> show parameter db_file_name_convert 

NAME                                 TYPE        VALUE
------------------------------------ ----------- ------------------------------
db_file_name_convert                 string      +ABCE/test/DATAFILE/, /t01/app/oracle/oradata/teststby/datafile/, 
						 +ABCE/test/TEMPFILE/, /t01/app/oracle/oradata/teststby/tempfile/
SQL> alter system set standby_file_management=manual;

System altered.

SQL> alter database create datafile '/t01/app/oracle/testuct/11.2.0/dbhome_1/dbs/UNNAMED00008' as '/t01/app/oracle/oradata/teststby/datafile/dtest.295.981369995';

Database altered.

SQL> alter system set standby_file_management=auto;

　　

然后重启MRP

SQL> alter database recover managed standby database cancel;
alter database recover managed standby database cancel
*
ERROR at line 1:
ORA-16136: Managed Standby Recovery not active

SQL> alter database recover managed standby database disconnect from session;

　　

可以再次查看alert日志

Fri Jul 13 11:19:53 2018
alter database recover managed standby database cancel
ORA-16136 signalled during: alter database recover managed standby database cancel...
Fri Jul 13 11:21:06 2018
alter database recover managed standby database disconnect from session
Attempt to start background Managed Standby Recovery process (teststby)
Fri Jul 13 11:21:07 2018
MRP0 started with pid=26, OS id=10385 
MRP0: Background Managed Standby Recovery process started (teststby)
 started logmerger process
Fri Jul 13 11:21:12 2018
Managed Standby Recovery not using Real Time Apply
Parallel Media Recovery started with 4 slaves
Waiting for all non-current ORLs to be archived...
All non-current ORLs have been archived.
Media Recovery Log /t01/app/oracle/arch/2_5487_930568019.dbf
Media Recovery Log /t01/app/oracle/arch/1_7110_930568019.dbf
Completed: alter database recover managed standby database disconnect from session
Media Recovery Log /t01/app/oracle/arch/1_7111_930568019.dbf
Media Recovery Log /t01/app/oracle/arch/2_5488_930568019.dbf
Media Recovery Waiting for thread 2 sequence 5491 (in transit)