zoukankan      html  css  js  c++  java

  • oracle数据库启动报错，不能启动ASM实例

    数据库rac启动时报错，日志例如以下，后来使用

    Sat Jun  7 06:02:11 2014
    GATHER_STATS_JOB encountered errors.  Check the trace file.
    Sat Jun  7 06:02:11 2014
    Errors in file /oracle/product/admin/dqb/bdump/dqb2_j001_13352.trc:
    ORA-08103: object no longer exists
    Sat Jun  7 06:02:51 2014
    Thread 2 advanced to log sequence 13292 (LGWR switch)
      Current log# 4 seq# 13292 mem# 0: +DG/dqb/onlinelog/group_4.266.803845083
    Sat Jun  7 06:32:10 2014
    Trace dumping is performing id=[cdmp_20140607063209]
    Sat Jun  7 07:01:33 2014
    Errors in file /oracle/product/admin/dqb/bdump/dqb2_j000_6797.trc:
    ORA-00600: internal error code, arguments: [6749], [3], [12662723], [109], [], [], [], []
    Sat Jun  7 07:01:35 2014
    Trace dumping is performing id=[cdmp_20140607070135]
    Sat Jun  7 07:39:01 2014
    Errors in file /oracle/product/admin/dqb/udump/dqb2_ora_24893.trc:
    ORA-00600: internal error code, arguments: [kdsgrp1], [], [], [], [], [], [], []
    Sat Jun  7 07:39:04 2014
    Trace dumping is performing id=[cdmp_20140607073904]
    Sat Jun  7 08:00:42 2014
    Trace dumping is performing id=[cdmp_20140607080042]
    Sat Jun  7 08:30:13 2014
    Trace dumping is performing id=[cdmp_20140607083013]
    Sat Jun  7 09:00:43 2014
    Trace dumping is performing id=[cdmp_20140607090043]
    Sat Jun  7 09:36:24 2014
    Trace dumping is performing id=[cdmp_20140607093623]
    Sat Jun  7 10:00:46 2014
    Trace dumping is performing id=[cdmp_20140607100046]
    Sat Jun  7 10:38:31 2014
    Trace dumping is performing id=[cdmp_20140607103831]
    Sat Jun  7 11:00:52 2014
    Trace dumping is performing id=[cdmp_20140607110051]
    Sat Jun  7 11:37:40 2014
    Errors in file /oracle/product/admin/dqb/udump/dqb2_ora_18978.trc:
    ORA-00600: internal error code, arguments: [kdsgrp1], [], [], [], [], [], [], []
    Sat Jun  7 11:37:43 2014
    Trace dumping is performing id=[cdmp_20140607113743]
    Sat Jun  7 12:00:51 2014
    Trace dumping is performing id=[cdmp_20140607120050]
    Sat Jun  7 12:38:58 2014
    Trace dumping is performing id=[cdmp_20140607123858]
    Sat Jun  7 13:00:56 2014
    Trace dumping is performing id=[cdmp_20140607130056]
    Sat Jun  7 13:39:25 2014
    Errors in file /oracle/product/admin/dqb/udump/dqb2_ora_31749.trc:
    ORA-00600: internal error code, arguments: [kdsgrp1], [], [], [], [], [], [], []
    Sat Jun  7 13:39:28 2014
    Trace dumping is performing id=[cdmp_20140607133928]
    Sat Jun  7 13:44:08 2014
    Thread 2 advanced to log sequence 13293 (LGWR switch)
      Current log# 3 seq# 13293 mem# 0: +DG/dqb/onlinelog/group_3.268.803845083
    Sat Jun  7 14:00:59 2014
    Trace dumping is performing id=[cdmp_20140607140058]
    Sat Jun  7 14:38:47 2014
    Errors in file /oracle/product/admin/dqb/udump/dqb2_ora_5659.trc:
    ORA-00600: internal error code, arguments: [kdsgrp1], [], [], [], [], [], [], []
    Sat Jun  7 14:38:48 2014
    Trace dumping is performing id=[cdmp_20140607143848]
    Sat Jun  7 15:00:44 2014
    Trace dumping is performing id=[cdmp_20140607150043]
    Sat Jun  7 15:34:03 2014
    Errors in file /oracle/product/admin/dqb/udump/dqb2_ora_12186.trc:
    ORA-00600: internal error code, arguments: [kdsgrp1], [], [], [], [], [], [], []
    Sat Jun  7 15:34:05 2014
    Trace dumping is performing id=[cdmp_20140607153405]
    Sat Jun  7 15:43:52 2014
    Thread 2 advanced to log sequence 13294 (LGWR switch)
      Current log# 4 seq# 13294 mem# 0: +DG/dqb/onlinelog/group_4.266.803845083
    Sat Jun  7 16:02:14 2014
    Errors in file /oracle/product/admin/dqb/bdump/dqb2_j001_13352.trc:
    ORA-00600: internal error code, arguments: [6749], [3], [12662723], [109], [], [], [], []
    Sat Jun  7 16:02:19 2014
    Trace dumping is performing id=[cdmp_20140607160219]
    Sat Jun  7 16:03:35 2014
    Thread 2 advanced to log sequence 13295 (LGWR switch)
      Current log# 3 seq# 13295 mem# 0: +DG/dqb/onlinelog/group_3.268.803845083
    Sat Jun  7 16:40:00 2014
    Trace dumping is performing id=[cdmp_20140607164000]
    Sat Jun  7 17:01:30 2014
    Errors in file /oracle/product/admin/dqb/bdump/dqb2_j001_13352.trc:
    ORA-00600: internal error code, arguments: [6749], [3], [12662723], [109], [], [], [], []
    Sat Jun  7 17:01:33 2014
    Trace dumping is performing id=[cdmp_20140607170133]
    Sat Jun  7 17:32:37 2014
    Trace dumping is performing id=[cdmp_20140607173236]
    Sat Jun  7 18:01:51 2014
    Errors in file /oracle/product/admin/dqb/bdump/dqb2_j001_13352.trc:
    ORA-00600: internal error code, arguments: [6749], [3], [12662723], [109], [], [], [], []
    Sat Jun  7 18:01:53 2014
    Trace dumping is performing id=[cdmp_20140607180153]
    Sat Jun  7 18:29:15 2014
    Thread 2 advanced to log sequence 13296 (LGWR switch)
      Current log# 4 seq# 13296 mem# 0: +DG/dqb/onlinelog/group_4.266.803845083
    Sat Jun  7 18:30:52 2014
    Errors in file /oracle/product/admin/dqb/udump/dqb2_ora_19302.trc:
    Sat Jun  7 18:30:54 2014
    Trace dumping is performing id=[cdmp_20140607183054]
    Sat Jun  7 18:30:58 2014
    Errors in file /oracle/product/admin/dqb/udump/dqb2_ora_19300.trc:
    Sat Jun  7 18:31:05 2014
    Errors in file /oracle/product/admin/dqb/udump/dqb2_ora_19300.trc:
    Sat Jun  7 18:42:14 2014
    Errors in file /oracle/product/admin/dqb/udump/dqb2_ora_31207.trc:
    ORA-00600: internal error code, arguments: [kdsgrp1], [], [], [], [], [], [], []
    Sat Jun  7 18:42:20 2014
    Trace dumping is performing id=[cdmp_20140607184220]
    Sat Jun  7 19:00:04 2014
    Trace dumping is performing id=[cdmp_20140607190003]
    Sat Jun  7 19:31:17 2014
    Errors in file /oracle/product/admin/dqb/udump/dqb2_ora_4965.trc:
    ORA-00600: internal error code, arguments: [kdsgrp1], [], [], [], [], [], [], []
    Sat Jun  7 19:31:18 2014
    Trace dumping is performing id=[cdmp_20140607193118]
    Sat Jun  7 20:00:42 2014
    Trace dumping is performing id=[cdmp_20140607200042]
    Sat Jun  7 20:31:30 2014
    Errors in file /oracle/product/admin/dqb/udump/dqb2_ora_11315.trc:
    ORA-00600: internal error code, arguments: [kdsgrp1], [], [], [], [], [], [], []
    Sat Jun  7 20:31:32 2014
    Trace dumping is performing id=[cdmp_20140607203132]
    Sat Jun  7 21:33:02 2014
    Trace dumping is performing id=[cdmp_20140607213302]
    Sat Jun  7 22:02:16 2014
    Errors in file /oracle/product/admin/dqb/bdump/dqb2_j000_6797.trc:
    ORA-00600: internal error code, arguments: [6749], [3], [12662723], [109], [], [], [], []
    Sat Jun  7 22:17:00 2014
    Thread 2 advanced to log sequence 13297 (LGWR switch)
      Current log# 3 seq# 13297 mem# 0: +DG/dqb/onlinelog/group_3.268.803845083
    Sat Jun  7 22:30:38 2014
    Trace dumping is performing id=[cdmp_20140607223037]
    Sat Jun  7 23:01:21 2014
    Errors in file /oracle/product/admin/dqb/bdump/dqb2_j001_13352.trc:
    ORA-00600: internal error code, arguments: [6749], [3], [12662723], [109], [], [], [], []
    Sat Jun  7 23:01:25 2014
    Trace dumping is performing id=[cdmp_20140607230125]
    Sat Jun  7 23:30:33 2014
    Trace dumping is performing id=[cdmp_20140607233032]
    Sun Jun  8 00:00:38 2014
    Trace dumping is performing id=[cdmp_20140608000038]
    Sun Jun  8 00:39:30 2014
    Trace dumping is performing id=[cdmp_20140608003930]
    Sun Jun  8 01:00:36 2014
    Trace dumping is performing id=[cdmp_20140608010036]
    Sun Jun  8 01:45:14 2014
    Trace dumping is performing id=[cdmp_20140608014514]
    Sun Jun  8 02:00:41 2014
    Trace dumping is performing id=[cdmp_20140608020041]
    Sun Jun  8 02:36:03 2014
    Trace dumping is performing id=[cdmp_20140608023603]
    Sun Jun  8 03:31:51 2014
    Errors in file /oracle/product/admin/dqb/udump/dqb2_ora_24887.trc:
    ORA-00600: internal error code, arguments: [kdsgrp1], [], [], [], [], [], [], []
    Sun Jun  8 03:31:54 2014
    Trace dumping is performing id=[cdmp_20140608033154]
    Sun Jun  8 04:05:37 2014
    >>> WAITED TOO LONG FOR A ROW CACHE ENQUEUE LOCK! pid=54
    System State dumped to trace file /oracle/product/admin/dqb/bdump/dqb2_mmon_1398.trc
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:15 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 04:47:20 2014
    Trace dumping is performing id=[cdmp_20140608044719]
    Sun Jun  8 05:00:22 2014
    Trace dumping is performing id=[cdmp_20140608050022]
    Sun Jun  8 05:39:23 2014
    Trace dumping is performing id=[cdmp_20140608053922]
    Sun Jun  8 06:00:28 2014
    Trace dumping is performing id=[cdmp_20140608060027]
    Sun Jun  8 06:32:45 2014
    Trace dumping is performing id=[cdmp_20140608063245]
    Sun Jun  8 07:00:30 2014
    Trace dumping is performing id=[cdmp_20140608070030]
    Sun Jun  8 07:11:17 2014
    Thread 2 advanced to log sequence 13298 (LGWR switch)
      Current log# 4 seq# 13298 mem# 0: +DG/dqb/onlinelog/group_4.266.803845083
    Sun Jun  8 07:30:12 2014
    Errors in file /oracle/product/admin/dqb/udump/dqb2_ora_6955.trc:
    ORA-00600: internal error code, arguments: [kdsgrp1], [], [], [], [], [], [], []
    Sun Jun  8 07:30:15 2014
    Trace dumping is performing id=[cdmp_20140608073015]
    Sun Jun  8 08:00:32 2014
    Trace dumping is performing id=[cdmp_20140608080031]
    Sun Jun  8 08:30:04 2014
    Trace dumping is performing id=[cdmp_20140608083004]
    Sun Jun  8 09:36:04 2014
    Trace dumping is performing id=[cdmp_20140608093604]
    Sun Jun  8 10:00:27 2014
    Trace dumping is performing id=[cdmp_20140608100026]
    Sun Jun  8 10:35:29 2014
    Trace dumping is performing id=[cdmp_20140608103528]
    Sun Jun  8 11:00:32 2014
    Trace dumping is performing id=[cdmp_20140608110031]
    Sun Jun  8 11:31:07 2014
    Errors in file /oracle/product/admin/dqb/udump/dqb2_ora_32192.trc:
    ORA-00600: internal error code, arguments: [kdsgrp1], [], [], [], [], [], [], []
    Sun Jun  8 11:31:13 2014
    Trace dumping is performing id=[cdmp_20140608113113]
    Sun Jun  8 12:39:37 2014
    Errors in file /oracle/product/admin/dqb/udump/dqb2_ora_5862.trc:
    ORA-00600: internal error code, arguments: [kdsgrp1], [], [], [], [], [], [], []
    Sun Jun  8 12:39:38 2014
    Trace dumping is performing id=[cdmp_20140608123938]
    Sun Jun  8 13:00:20 2014
    Trace dumping is performing id=[cdmp_20140608130020]
    Sun Jun  8 13:40:14 2014
    Trace dumping is performing id=[cdmp_20140608134014]
    Sun Jun  8 14:00:21 2014
    Trace dumping is performing id=[cdmp_20140608140020]
    Sun Jun  8 14:35:35 2014
    Trace dumping is performing id=[cdmp_20140608143534]
    Sun Jun  8 15:34:51 2014
    >>> WAITED TOO LONG FOR A ROW CACHE ENQUEUE LOCK! pid=54
    System State dumped to trace file /oracle/product/admin/dqb/bdump/dqb2_mmon_1398.trc
    Sun Jun  8 15:37:47 2014
    Errors in file /oracle/product/admin/dqb/udump/dqb2_ora_25433.trc:
    ORA-00600: internal error code, arguments: [kdsgrp1], [], [], [], [], [], [], []
    Sun Jun  8 15:37:50 2014
    Trace dumping is performing id=[cdmp_20140608153750]
    Sun Jun  8 16:13:50 2014
    >>> WAITED TOO LONG FOR A ROW CACHE ENQUEUE LOCK! pid=121
    System State dumped to trace file /oracle/product/admin/dqb/udump/dqb2_ora_11844.trc
    Sun Jun  8 16:58:04 2014
    >>> WAITED TOO LONG FOR A ROW CACHE ENQUEUE LOCK! pid=136
    System State dumped to trace file /oracle/product/admin/dqb/udump/dqb2_ora_25750.trc
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:19 2014
    WARNING: inbound connection timed out (ORA-3136)
    Sun Jun  8 21:36:21 2014
    Trace dumping is performing id=[cdmp_20140608213621]
    Sun Jun  8 21:36:22 2014
    Thread 2 advanced to log sequence 13299 (LGWR switch)
      Current log# 3 seq# 13299 mem# 0: +DG/dqb/onlinelog/group_3.268.803845083
    Sun Jun  8 22:00:15 2014
    Trace dumping is performing id=[cdmp_20140608220015]
    Sun Jun  8 22:40:03 2014
    Errors in file /oracle/product/admin/dqb/udump/dqb2_ora_22234.trc:
    ORA-00600: internal error code, arguments: [kdsgrp1], [], [], [], [], [], [], []
    Sun Jun  8 22:40:06 2014
    Trace dumping is performing id=[cdmp_20140608224006]
    Sun Jun  8 23:00:18 2014
    Trace dumping is performing id=[cdmp_20140608230018]
    Sun Jun  8 23:29:52 2014
    Errors in file /oracle/product/admin/dqb/udump/dqb2_ora_27653.trc:
    ORA-00600: internal error code, arguments: [kdsgrp1], [], [], [], [], [], [], []
    Sun Jun  8 23:29:53 2014
    Trace dumping is performing id=[cdmp_20140608232953]
    Mon Jun  9 00:44:11 2014
    Errors in file /oracle/product/admin/dqb/udump/dqb2_ora_30862.trc:
    ORA-00600: internal error code, arguments: [kdsgrp1], [], [], [], [], [], [], []
    Mon Jun  9 00:44:15 2014
    Trace dumping is performing id=[cdmp_20140609004415]
    Mon Jun  9 01:24:41 2014
    >>> WAITED TOO LONG FOR A ROW CACHE ENQUEUE LOCK! pid=174
    System State dumped to trace file /oracle/product/admin/dqb/udump/dqb2_ora_13953.trc
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:22 2014
    WARNING: inbound connection timed out (ORA-3136)
    Mon Jun  9 05:56:24 2014
    Trace dumping is performing id=[cdmp_20140609055624]
    Mon Jun  9 05:57:37 2014
    Thread 2 advanced to log sequence 13300 (LGWR switch)
      Current log# 4 seq# 13300 mem# 0: +DG/dqb/onlinelog/group_4.266.803845083
    Mon Jun  9 06:07:55 2014
    Errors in file /oracle/product/admin/dqb/bdump/dqb2_j006_15568.trc:
    ORA-12012: error on auto execute of job 524
    ORA-06502: PL/SQL: numeric or value error: character string buffer too small
    ORA-06512: at "ZDDX_NEW.PKG_BB_LKZHFJTJ_W", line 537
    ORA-06512: at "ZDDX_NEW.PKG_BB_LKZHFJTJ_W", line 9
    ORA-06512: at "ZDDX_NEW.PKG_BB_TOOL_W", line 309
    ORA-06512: at line 1
    Mon Jun  9 06:41:35 2014
    Thread 2 advanced to log sequence 13301 (LGWR switch)
      Current log# 3 seq# 13301 mem# 0: +DG/dqb/onlinelog/group_3.268.803845083
    Mon Jun  9 06:57:53 2014
    >>> WAITED TOO LONG FOR A ROW CACHE ENQUEUE LOCK! pid=54
    System State dumped to trace file /oracle/product/admin/dqb/bdump/dqb2_mmon_1398.trc
    Mon Jun  9 07:24:19 2014
    >>> WAITED TOO LONG FOR A ROW CACHE ENQUEUE LOCK! pid=174
    System State dumped to trace file /oracle/product/admin/dqb/udump/dqb2_ora_1963.trc
    Mon Jun  9 09:45:09 2014
    Trace dumping is performing id=[cdmp_20140609094508]
    Mon Jun  9 09:45:36 2014
    Errors in file /oracle/product/admin/dqb/bdump/dqb2_asmb_1428.trc:
    ORA-15064: communication failure with ASM instance
    ORA-03113: end-of-file on communication channel
    Mon Jun  9 09:45:36 2014
    ASMB: terminating instance due to error 15064
    Mon Jun  9 09:45:36 2014
    Errors in file /oracle/product/admin/dqb/bdump/dqb2_lmon_1337.trc:
    ORA-15064: communication failure with ASM instance
    Mon Jun  9 09:45:36 2014
    Errors in file /oracle/product/admin/dqb/bdump/dqb2_pmon_1331.trc:
    ORA-15064: communication failure with ASM instance
    Mon Jun  9 09:45:36 2014
    Errors in file /oracle/product/admin/dqb/bdump/dqb2_lms5_1361.trc:
    ORA-15064: communication failure with ASM instance
    Mon Jun  9 09:45:36 2014
    Errors in file /oracle/product/admin/dqb/bdump/dqb2_lms1_1345.trc:
    ORA-15064: communication failure with ASM instance
    Mon Jun  9 09:45:36 2014
    Errors in file /oracle/product/admin/dqb/bdump/dqb2_arc1_1545.trc:
    ORA-00600: internal error code, arguments: [kfnDisconnect], [not connected], [], [], [], [], [], []
    ORA-15055: unable to connect to ASM instance
    ORA-01092: ORACLE instance terminated. Disconnection forced
    ORA-03113: end-of-file on communication channel
    Mon Jun  9 09:45:36 2014
    Errors in file /oracle/product/admin/dqb/bdump/dqb2_lms2_1349.trc:
    ORA-15064: communication failure with ASM instance
    Mon Jun  9 09:45:36 2014
    Errors in file /oracle/product/admin/dqb/bdump/dqb2_lms6_1365.trc:
    ORA-15064: communication failure with ASM instance
    Mon Jun  9 09:45:36 2014
    Errors in file /oracle/product/admin/dqb/bdump/dqb2_lms0_1341.trc:
    ORA-15064: communication failure with ASM instance
    Mon Jun  9 09:45:36 2014
    Errors in file /oracle/product/admin/dqb/bdump/dqb2_lms7_1369.trc:
    ORA-15064: communication failure with ASM instance
    Mon Jun  9 09:45:36 2014
    Errors in file /oracle/product/admin/dqb/bdump/dqb2_lms3_1353.trc:
    ORA-15064: communication failure with ASM instance
    Mon Jun  9 09:45:36 2014
    Errors in file /oracle/product/admin/dqb/bdump/dqb2_lms4_1357.trc:
    ORA-15064: communication failure with ASM instance
    Mon Jun  9 09:45:36 2014
    Errors in file /oracle/product/admin/dqb/bdump/dqb2_lmd0_1339.trc:
    ORA-15064: communication failure with ASM instance
    Mon Jun  9 09:45:37 2014
    Errors in file /oracle/product/admin/dqb/bdump/dqb2_arc1_1545.trc:
    ORA-00600: internal error code, arguments: [kfnDisconnect], [not connected], [], [], [], [], [], []
    ORA-15055: unable to connect to ASM instance
    ORA-01092: ORACLE instance terminated. Disconnection forced
    ORA-03113: end-of-file on communication channel
    Mon Jun  9 09:45:37 2014
    Errors in file /oracle/product/admin/dqb/bdump/dqb2_arc1_1545.trc:
    ORA-00600: internal error code, arguments: [kfnDisconnect], [not connected], [], [], [], [], [], []
    ORA-15055: unable to connect to ASM instance
    ORA-01092: ORACLE instance terminated. Disconnection forced
    ORA-03113: end-of-file on communication channel
    Mon Jun  9 09:45:37 2014
    Errors in file /oracle/product/admin/dqb/bdump/dqb2_arc1_1545.trc:
    ORA-00600: internal error code, arguments: [kfnDisconnect], [not connected], [], [], [], [], [], []
    ORA-15055: unable to connect to ASM instance
    ORA-01092: ORACLE instance terminated. Disconnection forced
    ORA-03113: end-of-file on communication channel
    Mon Jun  9 09:45:37 2014
    Errors in file /oracle/product/admin/dqb/bdump/dqb2_arc1_1545.trc:
    ORA-00600: internal error code, arguments: [kfnDisconnect], [not connected], [], [], [], [], [], []
    ORA-15055: unable to connect to ASM instance
    ORA-01092: ORACLE instance terminated. Disconnection forced
    ORA-03113: end-of-file on communication channel
    Mon Jun  9 09:45:37 2014
    Errors in file /oracle/product/admin/dqb/bdump/dqb2_arc1_1545.trc:
    ORA-00600: internal error code, arguments: [kfnDisconnect], [not connected], [], [], [], [], [], []
    ORA-15055: unable to connect to ASM instance
    ORA-01092: ORACLE instance terminated. Disconnection forced
    ORA-03113: end-of-file on communication channel
    Mon Jun  9 09:45:37 2014
    Errors in file /oracle/product/admin/dqb/bdump/dqb2_arc1_1545.trc:
    ORA-00600: internal error code, arguments: [kfnDisconnect], [not connected], [], [], [], [], [], []
    ORA-15055: unable to connect to ASM instance
    ORA-01092: ORACLE instance terminated. Disconnection forced
    ORA-03113: end-of-file on communication channel
    Mon Jun  9 09:45:38 2014
    Errors in file /oracle/product/admin/dqb/bdump/dqb2_rbal_1432.trc:
    ORA-15064: communication failure with ASM instance
    Mon Jun  9 09:45:39 2014
    Trace dumping is performing id=[cdmp_20140609094539]
    Mon Jun  9 09:45:47 2014
    Instance terminated by ASMB, pid = 1428
    Mon Jun  9 09:53:01 2014
    Starting ORACLE instance (normal)
    LICENSE_MAX_SESSION = 0
    LICENSE_SESSIONS_WARNING = 0
    Interface type 1 eth1 192.168.1.0 configured from OCR for use as a cluster interconnect
    Interface type 1 eth0 10.177.1.0 configured from OCR for use as  a public interface
    Picked latch-free SCN scheme 3
    Autotune of undo retention is turned on.
    LICENSE_MAX_USERS = 0
    SYS auditing is disabled
    ksdpec: called for event 13740 prior to event group initialization
    Starting up ORACLE RDBMS Version: 10.2.0.4.0.
    System parameters with non-default values:
      processes                = 300
      sessions                 = 335
      __shared_pool_size       = 452984832
      __large_pool_size        = 16777216
      __java_pool_size         = 16777216
      __streams_pool_size      = 0
      spfile                   = +DG/dqb/spfiledqb.ora
      sga_target               = 1610612736
      control_files            = +DG/dqb/controlfile/current.272.803844991
      control_file_record_keep_time= 8
      db_block_size            = 8192
      __db_cache_size          = 1107296256
      compatible               = 10.2.0.3.0
      log_archive_dest_1       = LOCATION=+dg/dqb/arch/arch1
      db_file_multiblock_read_count= 16
      cluster_database         = TRUE
      cluster_database_instances= 2
      db_create_file_dest      = +DG
      db_recovery_file_dest_size= 10737418240
      thread                   = 2
      instance_number          = 2
      undo_management          = AUTO
      undo_tablespace          = UNDOTBS4
      remote_login_passwordfile= EXCLUSIVE
      db_domain                =
      dispatchers              = (PROTOCOL=TCP) (SERVICE=dqbXDB)
      remote_listener          = LISTENERS_DQB
      job_queue_processes      = 10
      background_dump_dest     = /oracle/product/admin/dqb/bdump
      user_dump_dest           = /oracle/product/admin/dqb/udump
      core_dump_dest           = /oracle/product/admin/dqb/cdump
      audit_file_dest          = /oracle/product/admin/dqb/adump
      db_name                  = dqb
      open_cursors             = 300
      pga_aggregate_target     = 3363831808
    Cluster communication is configured to use the following interface(s) for this instance
      192.168.1.2
    Mon Jun  9 09:53:01 2014
    cluster interconnect IPC version:Oracle UDP/IP (generic)
    IPC Vendor 1 proto 2
    PMON started with pid=4, OS id=13866
    DIAG started with pid=8, OS id=13868
    PSP0 started with pid=12, OS id=13870
    LMON started with pid=16, OS id=13872
    LMD0 started with pid=2, OS id=13874
    LMS0 started with pid=6, OS id=13876
    LMS1 started with pid=3, OS id=13880
    LMS2 started with pid=20, OS id=13884
    LMS3 started with pid=5, OS id=13888
    LMS4 started with pid=10, OS id=13892
    LMS5 started with pid=7, OS id=13896
    LMS6 started with pid=24, OS id=13900
    LMS7 started with pid=9, OS id=13904
    MMAN started with pid=28, OS id=13908
    DBW0 started with pid=14, OS id=13910
    DBW1 started with pid=11, OS id=13912
    DBW2 started with pid=32, OS id=13914
    DBW3 started with pid=13, OS id=13916
    LGWR started with pid=15, OS id=13918
    CKPT started with pid=36, OS id=13920
    SMON started with pid=40, OS id=13922
    RECO started with pid=44, OS id=13924
    CJQ0 started with pid=18, OS id=13926
    MMON started with pid=48, OS id=13928
    Mon Jun  9 09:53:02 2014
    starting up 1 dispatcher(s) for network address '(ADDRESS=(PARTIAL=YES)(PROTOCOL=TCP))'...
    MMNL started with pid=52, OS id=13936
    Mon Jun  9 09:53:02 2014
    starting up 1 shared server(s) ...
    Mon Jun  9 09:53:02 2014
    lmon registered with NM - instance id 2 (internal mem no 1)
    Mon Jun  9 09:53:02 2014
    Reconfiguration started (old inc 0, new inc 2)
    List of nodes:
     1
     Global Resource Directory frozen
    * allocate domain 0, invalid = TRUE
     Communication channels reestablished
     Master broadcasted resource hash value bitmaps
     Non-local Process blocks cleaned out
    Mon Jun  9 09:53:02 2014
     LMS 2: 0 GCS shadows cancelled, 0 closed
    Mon Jun  9 09:53:02 2014
     LMS 0: 0 GCS shadows cancelled, 0 closed
    Mon Jun  9 09:53:02 2014
     LMS 3: 0 GCS shadows cancelled, 0 closed
    Mon Jun  9 09:53:02 2014
     LMS 1: 0 GCS shadows cancelled, 0 closed
    Mon Jun  9 09:53:02 2014
     LMS 6: 0 GCS shadows cancelled, 0 closed
    Mon Jun  9 09:53:02 2014
     LMS 7: 0 GCS shadows cancelled, 0 closed
    Mon Jun  9 09:53:02 2014
     LMS 5: 0 GCS shadows cancelled, 0 closed
    Mon Jun  9 09:53:02 2014
     LMS 4: 0 GCS shadows cancelled, 0 closed
     Set master node info
     Submitted all remote-enqueue requests
     Dwn-cvts replayed, VALBLKs dubious
     All grantable enqueues granted
     Post SMON to start 1st pass IR
    Mon Jun  9 09:53:02 2014
     LMS 2: 0 GCS shadows traversed, 0 replayed
    Mon Jun  9 09:53:02 2014
     LMS 3: 0 GCS shadows traversed, 0 replayed
    Mon Jun  9 09:53:02 2014
     LMS 7: 0 GCS shadows traversed, 0 replayed
    Mon Jun  9 09:53:02 2014
     LMS 1: 0 GCS shadows traversed, 0 replayed
    Mon Jun  9 09:53:02 2014
     LMS 0: 0 GCS shadows traversed, 0 replayed
    Mon Jun  9 09:53:02 2014
     LMS 5: 0 GCS shadows traversed, 0 replayed
    Mon Jun  9 09:53:02 2014
     LMS 4: 0 GCS shadows traversed, 0 replayed
    Mon Jun  9 09:53:02 2014
     LMS 6: 0 GCS shadows traversed, 0 replayed
    Mon Jun  9 09:53:02 2014
     Submitted all GCS remote-cache requests
     Fix write in gcs resources
    Reconfiguration complete
    LCK0 started with pid=30, OS id=13942
    Mon Jun  9 09:53:03 2014
    ALTER DATABASE   MOUNT
    Mon Jun  9 09:53:03 2014
    This instance was first to mount
    Mon Jun  9 09:53:03 2014
    Starting background process ASMB
    ASMB started with pid=56, OS id=13953
    Starting background process RBAL
    RBAL started with pid=60, OS id=13957
    Mon Jun  9 09:53:06 2014
    SUCCESS: diskgroup DG was mounted
    Mon Jun  9 09:53:10 2014
    Setting recovery target incarnation to 2
    Mon Jun  9 09:53:11 2014
    Successful mount of redo thread 2, with mount id 4128760447
    Mon Jun  9 09:53:11 2014
    Database mounted in Shared Mode (CLUSTER_DATABASE=TRUE)
    Completed: ALTER DATABASE   MOUNT
    Mon Jun  9 09:53:11 2014
    ALTER DATABASE OPEN
    This instance was first to open
    Mon Jun  9 09:53:12 2014
    Beginning crash recovery of 2 threads
     parallel recovery started with 16 processes
    Mon Jun  9 09:53:14 2014
    Started redo scan
    Mon Jun  9 09:53:14 2014
    Completed redo scan
     26 redo blocks read, 8 data blocks need recovery
    Mon Jun  9 09:53:14 2014
    Started redo application at
     Thread 1: logseq 16276, block 199337, scn 13914083540861
     Thread 2: logseq 13301, block 55078
    Mon Jun  9 09:53:14 2014
    Recovery of Online Redo Log: Thread 1 Group 2 Seq 16276 Reading mem 0
      Mem# 0: +DG/dqb/onlinelog/group_2.275.803844993
    Mon Jun  9 09:53:14 2014
    Recovery of Online Redo Log: Thread 2 Group 3 Seq 13301 Reading mem 0
      Mem# 0: +DG/dqb/onlinelog/group_3.268.803845083
    Mon Jun  9 09:53:14 2014
    Completed redo application
    Mon Jun  9 09:53:14 2014
    Completed crash recovery at
     Thread 1: logseq 16276, block 199337, scn 13914083560862
     Thread 2: logseq 13301, block 55104, scn 13914083661134
     8 data blocks read, 8 data blocks written, 26 redo blocks read
    Attempt to switch log for thread 1 failed. reason: 3
    Picked broadcast on commit scheme to generate SCNs
    Mon Jun  9 09:53:14 2014
    LGWR: STARTING ARCH PROCESSES
    ARC0 started with pid=58, OS id=14055
    Mon Jun  9 09:53:14 2014
    ARC0: Archival started
    ARC1: Archival started
    LGWR: STARTING ARCH PROCESSES COMPLETE
    ARC1 started with pid=35, OS id=14062
    ARC1: Becoming the 'no FAL' ARCH
    ARC1: Becoming the 'no SRL' ARCH
    Mon Jun  9 09:53:14 2014
    ARC0: Becoming the heartbeat ARCH
    Mon Jun  9 09:53:17 2014
    Thread 2 advanced to log sequence 13302 (thread open)
    Thread 2 opened at log sequence 13302
      Current log# 4 seq# 13302 mem# 0: +DG/dqb/onlinelog/group_4.266.803845083
    Successful open of redo thread 2
    Mon Jun  9 09:53:17 2014
    MTTR advisory is disabled because FAST_START_MTTR_TARGET is not set
    Mon Jun  9 09:53:17 2014
    SMON: enabling cache recovery
    Mon Jun  9 09:53:18 2014
    ARC0: Switch failed
    Mon Jun  9 09:53:18 2014
    Successfully onlined Undo Tablespace 35.
    Mon Jun  9 09:53:18 2014
    SMON: enabling tx recovery
    Mon Jun  9 09:53:18 2014
    Database Characterset is ZHS16GBK
    Opening with internal Resource Manager plan
    where NUMA PG = 4, CPUs = 20
    replication_dependency_tracking turned off (no async multimaster replication found)
    Starting background process QMNC
    QMNC started with pid=62, OS id=14171
    Mon Jun  9 09:53:19 2014
    Thread 1 advanced to log sequence 16277 (archiver wakeup)
    Mon Jun  9 09:53:22 2014
    Completed: ALTER DATABASE OPEN
    Mon Jun  9 09:53:26 2014
    ALTER SYSTEM SET service_names='rac' SCOPE=MEMORY SID='dqb2';
    Mon Jun  9 10:01:02 2014
    Errors in file /oracle/product/admin/dqb/bdump/dqb2_j001_14197.trc:
    ORA-00600: internal error code, arguments: [6749], [3], [12662723], [109], [], [], [], []
    Mon Jun  9 10:01:06 2014
    Trace dumping is performing id=[cdmp_20140609100106]
    Mon Jun  9 10:03:24 2014
    ALTER SYSTEM SET service_names='rac','dqb' SCOPE=MEMORY SID='dqb2';
    叫同事手工启动ASM实例。一步步看报错信息
    export ORACLE_SID=+ASM1
    sqlplus / as sysdba
    sql>startup;
    提示ora-29701。

    客户报oracle数据库出现了ora-29701错误，查看oracle的 error message
    ORA-29701: unable to connect to Cluster Manager
    Cause: Connect to CM failed or timed out.
    Action: Verify that the CM was started. If the CM was not started, start it and then retry the database startup.
               If the CM died or is not responding, check the Oracle and CM trace files for errors.
           当时以为肯定是cluster出问题了，可是客户说查询都能够运行，就是运行create table语句就出现这个错误，怪了！ 后来通过查看数  据库的告警日志的过程中发现原来是$ORACLE_HOME的空间满了，通过删除没有的trace文件。数据库一切正常！

    汗， 后来让同事尝试使用手工启动ASM实例，提示Ora-29701，汗，原来是磁盘空间满
查看全文
  • 相关阅读:
    【转】PowerDesigner数据库视图同时显示Code和Name
    [转]BT原理分析
    异常机制及throw与throws的区别(转)
    BS与CS的联系与区别。
    ASP.NET和C#的区别/
    上百例Silverlight网站及演示汇总，供友参考
    Bing Maps进阶系列九：使用MapCruncher进行地图切片并集成进Bing Maps
    【Silverlight】Bing Maps学习系列（八）：使用Bing Maps Silverlight Control加载自己部署的Google Maps
    学习MAP 地图好地址
    Bing必应地图中国API
  • 原文地址：https://www.cnblogs.com/ldxsuanfa/p/10946057.html
    • Copyright © 2011-2022 走看看