查看rac资源信息时发现如下错误：# crsctl stat res -t
ora.shinfo.db
1 ONLINE INTERMEDIATE shdb1 Stuck Archiver
2 ONLINE INTERMEDIATE shdb2 Stuck Archiver
查看alert日志
ARCH: Archival stopped, error occurred. Will continue retrying
ORACLE Instance shinfo1 - Archival Error
ORA-16038: log 1 sequence# 61 cannot be archived
ORA-00742: Log read detects lost write in thread %d sequence %d block %d
ORA-00312: online log 1 thread 1: '+DATADG/shinfo/redo01.log'
Fri Apr 23 22:49:07 2021
ARCH: Archival stopped, error occurred. Will continue retrying
ORACLE Instance shinfo1 - Archival Error
ORA-16014: log 1 sequence# 61 not archived, no available destinations
ORA-00312: online log 1 thread 1: '+DATADG/shinfo/redo01.log'
Fri Apr 23 22:54:07 2021
ARC1: All Archive destinations made inactive due to error 742
ARC1: Closing local archive destination LOG_ARCHIVE_DEST_1: '/arch/arch_1_61_1055095452.arc' (error 742) (shinfo1)
查看rac资源时发现状态异常
# crsctl stat res -t
ora.shinfo.db
1 ONLINE INTERMEDIATE shdb1 Stuck Archiver
2 ONLINE INTERMEDIATE shdb2 Stuck Archiver
查看alert日志显示：
ORACLE Instance shinfo1 - Archival Error
ORA-16038: log 1 sequence# 61 cannot be archived
ORA-00742: Log read detects lost write in thread %d sequence %d block %d
ORA-00312: online log 1 thread 1: '+DATADG/shinfo/redo01.log'
ARCH: Archival stopped, error occurred. Will continue retrying
ORACLE Instance shinfo1 - Archival Error
ORA-16014: log 1 sequence# 61 not archived, no available destinations
ORA-00312: online log 1 thread 1: '+DATADG/shinfo/redo01.log'
根据上面的日志，可以清楚的得出，现在的问题是由于联机日志存在问题，无法进行归档动作，导致数据库状态异常。
此时，数据库仍可正常访问
set lines 2000
select * from v$log;
GROUP# THREAD# SEQUENCE# BYTES BLOCKSIZE MEMBERS ARCHIVED STATUS FIRST_CHANGE# FIRST_TIME NEXT_CHANGE# NEXT_TIME
---------- ---------- ---------- ---------- ---------- ---------- --- ---------------- ------------- ------------------ ------------ ------------------
1 1 61 209715200 512 1 NO INACTIVE 2109698 2021-04-2310:13:33 2197167 2021-04-2320:50:32
2 1 67 209715200 512 1 NO CURRENT 2281548 2021-04-2323:38:26 2.8147E+14
3 2 58 209715200 512 1 NO INACTIVE 2109700 2021-04-2310:13:33 2217295 2021-04-2320:59:47
4 2 63 209715200 512 1 NO CURRENT 2281600 2021-04-2323:39:03 2.8147E+14
5 1 66 209715200 512 1 NO INACTIVE 2277243 2021-04-2323:23:37 2281548 2021-04-2323:38:26
6 2 62 209715200 512 1 NO INACTIVE 2277246 2021-04-2323:24:17 2281600 2021-04-2323:39:03
6 rows selected.
可发现ARCHIVED状态全为NO
col MEMBER for a50
select * from v$logfile;
GROUP# TYPE MEMBER IS_
---------- ------- -------------------------------------------------- ---
1 ONLINE +DATADG/shinfo/redo01.log NO
2 ONLINE +DATADG/shinfo/redo02.log NO
5 ONLINE +DATADG/shinfo/redo05.log NO
3 ONLINE +DATADG/shinfo/redo03.log NO
4 ONLINE +DATADG/shinfo/redo04.log NO
6 ONLINE +DATADG/shinfo/redo06.log NO
6 rows selected.
此种问题大致有以下几种原因：
1、归档日志目录满了
遇到这类情况，一般都是迅速清理可删除的归档，比如这里在rman下手工清除1天前归档：
RMAN> delete noprompt archivelog all completed before 'sysdate -1';
2、异常重启数据库
解决方法：
根据alert日志信息执行
SQL> alter database clear unarchived logfile '+DATADG/shinfo/redo01.log';
SQL> alter system switch logfile;
直到执行alter system switch logfile;后，日志显示如下信息，则表示正常
Archiver process freed from errors. No longer stopped。
---start mount;
---alter system dump logfile '+DATADG/shinfo/redo01.log' validate;
set lines 200
col member for a50
select group#,member,status,type from v$logfile;
alter database clear unarchived logfile group 2;
SQL> alter database open;
需要重新做全备，因为之前的序列号已经不连续，旧的备份不可用
观察alert日志，出现
ARCH: Archival stopped, error occurred. Will continue retrying
ORACLE Instance shinfo1 - Archival Error
ORA-16014: log 2 sequence# 67 not archived, no available destinations
ORA-00312: online log 2 thread 1: '+DATADG/shinfo/redo02.log'
alter database clear unarchived logfile '+DATADG/shinfo/redo02.log';
直到执行alter system switch logfile;后，日志显示如下信息，则表示正常
Archiver process freed from errors. No longer stopped。
此时，节点1已经恢复正常
# crsctl stat res -t
ora.shinfo.db
1 ONLINE ONLINE shdb1 Open
2 ONLINE INTERMEDIATE shdb2 Stuck Archiver
SQL> set lines 2000
SQL> select * from v$log;
GROUP# THREAD# SEQUENCE# BYTES BLOCKSIZE MEMBERS ARCHIVED STATUS FIRST_CHANGE# FIRST_TIME NEXT_CHANGE# NEXT_TIME
---------- ---------- ---------- ---------- ---------- ---------- --- ---------------- ------------- ------------------ ------------ ------------------
1 1 71 209715200 512 1 YES INACTIVE 2290964 2021-04-2400:51:07 2291389 2021-04-2400:55:28
2 1 73 209715200 512 1 NO CURRENT 2291606 2021-04-2400:57:46 2.8147E+14
3 2 70 209715200 512 1 NO CURRENT 2291626 2021-04-2400:58:02 2.8147E+14
4 2 69 209715200 512 1 YES INACTIVE 2291602 2021-04-2400:57:45 2291626 2021-04-2400:58:02
5 1 72 209715200 512 1 YES INACTIVE 2291389 2021-04-2400:55:28 2291606 2021-04-2400:57:46
6 2 68 209715200 512 1 YES INACTIVE 2291502 2021-04-2400:56:44 2291602 2021-04-2400:57:45
6 rows selected.
可看到ARCHIVED字段已经存在YES的状态
以同样的方法处理节点2问题
处理完成后，查看rac资源状态
# crsctl stat res -t
ora.shinfo.db
1 ONLINE ONLINE shdb1 Open
2 ONLINE ONLINE shdb2 Open