zoukankan      html  css  js  c++  java
  • delete删除数据造成归档日志增加,操作系统空间不足导致数据库hang住

    业务需求,对日志表历史数据进行清理。历史表均很大,使用delete 操作删除90天前的数据。

    第一部分:快速删除数据

    SQL> alter table CC.F_LOG  parallel 4;
    SQL>alter session enable parallel dml;    
    SQL> delete  FROM CC.F_LOG  S WHERE S.CREATE_DATE>(SYSDATE-90);
    执行计划确定并行

    -----------------------------------------------
    | Id | Operation |
    -----------------------------------------------
    | 0 | DELETE STATEMENT |
    | 1 | PX COORDINATOR |
    | 2 | PX SEND QC (RANDOM) |
    | 3 | INDEX MAINTENANCE |
    | 4 | PX RECEIVE |
    | 5 | PX SEND RANGE |
    | 6 | DELETE |
    | 7 | BUFFER SORT |
    | 8 | PX RECEIVE |
    | 9 | PX SEND HASH (BLOCK ADDRESS)|
    |* 10 | INDEX RANGE SCAN |
    -----------------------------------------------

    SQL> select INST_ID,sid,serial#,USERNAME,STATUS,MACHINE,SQL_ID,EVENT,(sysdate-LOGON_TIME)*86400 as "s",LAST_CALL_ET

    from gv$session where status='ACTIVE' and username is not null;
    INST_ID SID SERIAL# USERNAME STATUS MACHINE SQL_ID EVENT s LAST_CALL_ET
    ------- ------ ------- ------------------- --------------- -------------------- ------------------------------ ------- ------------
    1 24   44857 SYS ACTIVE p01 6yn37jfu2y88k PX Deq Credit: send blkd 190 189
    1 584  28067 SYS ACTIVE p01 2ymxxw3mapxd9 SQL*Net message from client 196 0
    1 884  40483 SYS ACTIVE p01 6yn37jfu2y88k PX Deq: Table Q Normal 190 189
    1 888   8663 SYS ACTIVE p01 6yn37jfu2y88k PX Deq: Execute Reply 3679 212
    1 1157 9679  SYS ACTIVE p01 6yn37jfu2y88k PX Deq: Table Q Normal 190 189
    1 1162 10801 SYS ACTIVE p01 6yn37jfu2y88k direct path write temp 190 189
    1 1437 22187 SYS ACTIVE p01 6yn37jfu2y88k PX Deq Credit: send blkd 190 189
    1 1728 17227 SYS ACTIVE p01 6yn37jfu2y88k PX Deq Credit: send blkd 190 189
    1 1740 25759 SYS ACTIVE p01 6yn37jfu2y88k PX Deq: Table Q Normal 190 189
    9 rows selected.


    删除完毕还原并行1

    SQL> alter table CC.F_LOG parallel 1;

    第二部分:数据删除hang

    删除数据后发现,数据库会话数量越来越多,且delete操作session event
    log file switch (archiving needed)  等待啥?需要等待日志切换归档完成?  为啥等待日志切换归档这么长时间? 观察alert

    SQL> select INST_ID,sid,serial#,USERNAME,STATUS,MACHINE,SQL_ID,EVENT,(sysdate-LOGON_TIME)*86400 as "s",LAST_CALL_ET
    from gv$session where status='ACTIVE' and username is not null; INST_ID SID SERIAL# USERNAME STATUS MACHINE SQL_ID EVENT ------- ------ ------- -------------- ---------- -------------------- -------------------- ------------------------------ 1 597 51393 CXPT ACTIVE future-app1 fmmd0cjd0ghpf buffer busy waits 1 598 45595 ac ACTIVE pfmptom01.yutong.com 6d7v2nay5vm9n log file switch (archiving needed) 1 1429 41689 ac ACTIVE pfmptom02.yutong.com 3dn60pvzrqz0g buffer busy waits 1 1431 1279 CXPT ACTIVE future-app1 azakfbj8ss7f8 buffer busy waits 1 1449 1569 ac ACTIVE pfmptom01.yutong.com 7nbhsah5kw0q2 log file switch (archiving needed) 1 1453 25515 SYSTEM ACTIVE ZZYTYT-P610764 1xz9mk73crhu8 log file switch (archiving needed) 1 1458 38363 ac ACTIVE pfmptom02.yutong.com 3dn60pvzrqz0g buffer busy waits 1 1709 27961 ac ACTIVE pfmptom01.yutong.com d8ats3n1vrm7t enq: US - contention 1 1714 62631 ac ACTIVE pfmptom01.yutong.com 0jg5553p23hup enq: SQ - contention 1 1716 57967 CXPT ACTIVE future-app2 azakfbj8ss7f8 enq: US - contention 1 1730 47159 ac ACTIVE pfmptom02.yutong.com 3dn60pvzrqz0g buffer busy waits 1 1731 10129 ac ACTIVE pfmptom01.yutong.com btgxyzwsc5sd1 enq: US - contention 1 2009 42483 ac ACTIVE pfmpbak01 94cs4sv2zua0u log file switch (archiving needed) 1 2033 64813 ac ACTIVE pfmptom02.yutong.com 3dn60pvzrqz0g buffer busy waits 48 rows selected.

    观察ALERT日志,可以发现归档日志无法创建报错

    Examine archive trace files for archiving errors
    Current log# 8 seq# 41282 mem# 0: /oracle/oradata/fmpdb/redo08.log
    Tue Aug 13 19:41:12 2019
    ARC1: Encountered disk I/O error 19502
    ARC1: Closing local archive destination LOG_ARCHIVE_DEST_1: '/oracle/archivelog/fmpdb_1_41278_961338832.log' (error 19502) (fmpdb)
    ARC1: I/O error 19502 archiving log 4 to '/oracle/archivelog/fmpdb_1_41278_961338832.log'
    ARCH: Archival stopped, error occurred. Will continue retrying
    ORACLE Instance fmpdb - Archival Error
    ORA-16038: log 4 sequence# 41278 cannot be archived
    ORA-19502: write error on file "", block number (block size=)
    ORA-00312: online log 4 thread 1: '/oracle/oradata/fmpdb/redo04.log'
    Tue Aug 13 19:41:13 2019
    ARCH: Archival stopped, error occurred. Will continue retrying
    ORACLE Instance fmpdb - Archival Error
    ORA-16014: log 4 sequence# 41278 not archived, no available destinations
    ORA-00312: online log 4 thread 1: '/oracle/oradata/fmpdb/redo04.log'
    Tue Aug 13 19:44:28 2019
    minact-scn: useg scan erroring out with error e:12751
    Tue Aug 13 19:46:14 2019
    ARC1: Encountered disk I/O error 19502
    ARC1: Closing local archive destination LOG_ARCHIVE_DEST_1: '/oracle/archivelog/fmpdb_1_41278_961338832.log' (error 19502) (fmpdb)
    ARC1: I/O error 19502 archiving log 4 to '/oracle/archivelog/fmpdb_1_41278_961338832.log'
    ARCH: Archival stopped, error occurred. Will continue retrying
    ORACLE Instance fmpdb - Archival Error
    ORA-16038: log 4 sequence# 41278 cannot be archived
    ORA-19502: write error on file "", block number (block size=)
    ORA-00312: online log 4 thread 1: '/oracle/oradata/fmpdb/redo04.log'
    Tue Aug 13 19:46:14 2019
    ARCH: Archival stopped, error occurred. Will continue retrying
    ORACLE Instance fmpdb - Archival Error
    ORA-16014: log 4 sequence# 41278 not archived, no available destinations
    ORA-00312: online log 4 thread 1: '/oracle/oradata/fmpdb/redo04.log'
    Tue Aug 13 19:49:35 2019
    minact-scn: useg scan erroring out with error e:12751
    Tue Aug 13 19:51:15 2019
    ARC1: Encountered disk I/O error 19502
    ARC1: Closing local archive destination LOG_ARCHIVE_DEST_1: '/oracle/ARC1: I/O error 19502 archiving log 4 to

    '/oracle/archivelog/fmpdb_1_41278_961338832.log'
    ARCH: Archival stopped, error occurred. Will continue retrying
    ORACLE Instance fmpdb - Archival Error
    ORA-16038: log 4 sequence# 41278 cannot be archived
    ORA-19502: write error on file "", block number (block size=)
    ORA-00312: online log 4 thread 1: '/oracle/oradata/fmpdb/redo04.log'
    ARCH: Archival stopped, error occurred. Will continue retrying
    ORACLE Instance fmpdb - Archival Error
    ORA-16014: log 4 sequence# 41278 not archived, no available destinations
    ORA-00312: online log 4 thread 1: '/oracle/oradata/fmpdb/redo04.log'

    Tue Aug 13 19:56:16 2019
    Archiver process freed from errors. No longer stopped
    Tue Aug 13 19:56:18 2019
    Archived Log entry 41273 added for thread 1 sequence 41278 ID 0x67cfc4e dest 1:
    krse_arc_driver_core: Successful archiving of previously failed ORL
    Tue Aug 13 19:56:18 2019
    Thread 1 advanced to log sequence 41283 (LGWR switch)
    Current log# 4 seq# 41283 mem# 0: /oracle/oradata/fmpdb/redo04.log
    Tue Aug 13 19:56:18 2019
    Some DDE async actions failed or were cancelled
    Tue Aug 13 19:56:23 2019
    Archived Log entry 41274 added for thread 1 sequence 41281 ID 0x67cfc4e dest 1:
    Archived Log entry 41275 added for thread 1 sequence 41279 ID 0x67cfc4e dest 1:
    Archived Log entry 41276 added for thread 1 sequence 41280 ID 0x67cfc4e dest 1:
    Archived Log entry 41277 added for thread 1 sequence 41282 ID 0x67cfc4e dest 1:
    Tue Aug 13 19:56:43 2019
    Thread 1 advanced to log sequence 41284 (LGWR switch)
    Current log# 5 seq# 41284 mem# 0: /oracle/oradata/fmpdb/redo05.log
    Tue Aug 13 19:56:45 2019
    Archived Log entry 41278 added for thread 1 sequence 41283 ID 0x67cfc4e dest 1:
    Tue Aug 13 19:57:10 2019
    Thread 1 cannot allocate new log, sequence 41285

    操作手工删除归档日志后,快速执行脚本压缩备份归档日志删除已备份的归档日志释放空间。 delete操作恢复且执行完毕后,对数据库立即进行全备份。
    [root@pfmpodb01 ~]# df -h
    Filesystem Size Used Avail Use% Mounted on
    /dev/mapper/VolGroup00-LogVol00 297G 297G 0G 100% /

    清理归档日志
    [root@pfmpodb01 ~]# df -h
    Filesystem Size Used Avail Use% Mounted on
    /dev/mapper/VolGroup00-LogVol00 297G 271G 12G 97% /

    压缩备份归档日志后删除日志
    backup format '/oracle/dbbak/archbak_%d_%s_%p_%T' archivelog all delete input;

    [root@pfmpodb01 archivelog]# df -h
    Filesystem Size Used Avail Use% Mounted on
    /dev/mapper/VolGroup00-LogVol00 297G 230G 52G 82% /

    Resolving Issues Where 'log file switch (archiving needed)' Waits Occur Because Log has not yet been Archived (文档 ID 1476444.1)    
    
    Problem Confirmation:
    The time spent actively in the local database is significant
    Only certain sessions, queries or jobs are experiencing slowness (not throughout the database)
    Waits for 'log file switch (archiving needed)' are a significant component of DB Time
    Free space in archive destination is less than redolog file size
    Waits for 'log file switch (archiving needed)'
    This wait event occurs when waiting for a log switch because the log we will be switching into has not been archived yet.
    
    Reducing Waits
    Check the alert file to make sure that archiving has not stopped due to a failed archive write.
    To speed archiving, consider adding more archive processes or putting the archive files on striped disks.
    Check the archive log destination for space and make sure it is not full. If it is full, free up space.
  • 相关阅读:
    部署iis服务器与c#程序遇到的问题小结
    Installing node-oracledb on Microsoft Windows
    Navicat:cant create OCI environment.
    socket.io 中文手册 socket.io 中文文档
    现有的情绪
    Express 4.x Node.js的Web框架
    结构与类的对象在引用上的不同之处
    Android时间戳转换为标准Datetime(yyyy-MM-dd hh:mm:ss)格式
    ASP.NET MVC4中调用WEB API的四个方法
    安恒DASCTF 四月战 WP
  • 原文地址:https://www.cnblogs.com/lvcha001/p/11349637.html
Copyright © 2011-2022 走看看