一次oracle support的危机现场处理（之二）

最新推荐文章于 2024-04-29 08:51:00 发布
最新推荐文章于 2024-04-29 08:51:00 发布
阅读量5.9k 收藏 4
点赞数
分类专栏： ORACLE HPUNIX STORAGE 文章标签： oracle 2010 thread file 数据库 database
版权声明：本文为博主原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接和本声明。
本文链接：https://blog.csdn.net/csucxcc/article/details/5532896
版权
ORACLE 同时被 3 个专栏收录
106 篇文章 0 订阅
订阅专栏
STORAGE
24 篇文章 1 订阅
订阅专栏
HPUNIX
9 篇文章 0 订阅
订阅专栏

天有不测风云，

IBM 的中端磁盘阵列这次又惹祸了。在微码升级的时候，

DS4800 发生故障。导致当时在这台库上的几套生产库报错了。

IBM工程师把阵列故障恢复后，拍拍屁股走人，阵列上的数据又得我们来想办法抢救了。。。。

还好有oracle support的支持我们最终化解了这次危机。下文详细介绍了整个恢复的过程。

 

 

oracle工程师到达现场后，根据错误信息，
|||||||||||||||||||||||||||||||||||||||||||||||||||||
错误信息如下所示：
Mon Apr 26 20:40:06 2010
Starting ORACLE instance (normal)
LICENSE_MAX_SESSION = 0
LICENSE_SESSIONS_WARNING = 0
Picked latch-free SCN scheme 3
Using LOG_ARCHIVE_DEST_1 parameter default value as /oracle/product/10.2.0/db/dbs/arch
Autotune of undo retention is turned on.
IMODE=BR
ILAT =121
LICENSE_MAX_USERS = 0
SYS auditing is disabled
ksdpec: called for event 13740 prior to event group initialization
Starting up ORACLE RDBMS Version: 10.2.0.4.0.
System parameters with non-default values:
  processes                = 1000
  sessions                 = 1105
  sga_max_size             = 12884901888
  __shared_pool_size       = 5066719232
  __large_pool_size        = 16777216
  __java_pool_size         = 16777216
  __streams_pool_size      = 0
  sga_target               = 12884901888
  control_files            = /oradata/ccdev1/control01.ctl, /oradata/ccdev1/control02.ctl, /oradata/ccdev1/control03.ctl
  db_block_size            = 8192
  __db_cache_size          = 7767851008
  db_writer_processes      = 4
  compatible               = 10.2.0.3.0
  db_files                 = 300
  db_file_multiblock_read_count= 16
  undo_management          = AUTO
  undo_tablespace          = UNDOTBS1
  remote_login_passwordfile= EXCLUSIVE
  db_domain                =
  dispatchers              = (PROTOCOL=TCP) (SERVICE=ccdev1XDB)
  job_queue_processes      = 10
  background_dump_dest     = /oracle/admin/ccdev1/bdump
  user_dump_dest           = /oracle/admin/ccdev1/udump
  core_dump_dest           = /oracle/admin/ccdev1/cdump
  audit_file_dest          = /oracle/admin/ccdev1/adump
  db_name                  = ccdev1
  open_cursors             = 300
  pga_aggregate_target     = 4294967296
PMON started with pid=2, OS id=12592
PSP0 started with pid=3, OS id=12594
MMAN started with pid=4, OS id=12596
DBW0 started with pid=5, OS id=12598
DBW1 started with pid=6, OS id=12600
DBW2 started with pid=7, OS id=12602
DBW3 started with pid=8, OS id=12604
LGWR started with pid=9, OS id=12606
CKPT started with pid=10, OS id=12608
SMON started with pid=11, OS id=12610
RECO started with pid=12, OS id=12612
CJQ0 started with pid=13, OS id=12614
MMON started with pid=14, OS id=12616
Mon Apr 26 20:40:07 2010
starting up 1 dispatcher(s) for network address '(ADDRESS=(PARTIAL=YES)(PROTOCOL=TCP))'...
MMNL started with pid=15, OS id=12618
Mon Apr 26 20:40:07 2010
starting up 1 shared server(s) ...
Mon Apr 26 20:40:07 2010
ALTER DATABASE   MOUNT
Mon Apr 26 20:40:14 2010
Setting recovery target incarnation to 2
Mon Apr 26 20:40:14 2010
Successful mount of redo thread 1, with mount id 1621305962
Mon Apr 26 20:40:14 2010
Database mounted in Exclusive Mode
Completed: ALTER DATABASE   MOUNT
Mon Apr 26 20:40:14 2010
ALTER DATABASE OPEN
Mon Apr 26 20:40:14 2010
Beginning crash recovery of 1 threads
 parallel recovery started with 3 processes
Mon Apr 26 20:40:15 2010
Started redo scan
Mon Apr 26 20:40:15 2010
Completed redo scan
 37 redo blocks read, 6 data blocks need recovery
Mon Apr 26 20:40:15 2010
Started redo application at
 Thread 1: logseq 1340, block 3
Mon Apr 26 20:40:15 2010
Recovery of Online Redo Log: Thread 1 Group 2 Seq 1340 Reading mem 0
  Mem# 0: /oradata/ccdev1/redo02.log
Mon Apr 26 20:40:15 2010
Completed redo application
Mon Apr 26 20:40:15 2010
Completed crash recovery at
 Thread 1: logseq 1340, block 40, scn 9549479515357
 6 data blocks read, 6 data blocks written, 37 redo blocks read
Mon Apr 26 20:40:15 2010
Thread 1 advanced to log sequence 1341 (thread open)
Thread 1 opened at log sequence 1341
  Current log# 3 seq# 1341 mem# 0: /oradata/ccdev1/redo03.log
Successful open of redo thread 1
Mon Apr 26 20:40:15 2010
MTTR advisory is disabled because FAST_START_MTTR_TARGET is not set
Mon Apr 26 20:40:15 2010
SMON: enabling cache recovery
Mon Apr 26 20:40:15 2010
Successfully onlined Undo Tablespace 1.
Mon Apr 26 20:40:15 2010
SMON: enabling tx recovery
Mon Apr 26 20:40:15 2010
Database Characterset is ZHS16GBK
Opening with internal Resource Manager plan
where NUMA PG = 1, CPUs = 4
Mon Apr 26 20:40:15 2010
Errors in file /oracle/admin/ccdev1/bdump/ccdev1_smon_12610.trc:
ORA-00600: internal error code, arguments: [2663], [2223], [1767217797], [2223], [1767218866], [], [], []
Mon Apr 26 20:40:17 2010
Errors in file /oracle/admin/ccdev1/bdump/ccdev1_smon_12610.trc:
ORA-01595: error freeing extent (10) of rollback segment (1))
ORA-00600: internal error code, arguments: [2663], [2223], [1767217797], [2223], [1767218866], [], [], []
Mon Apr 26 20:40:17 2010
Errors in file /oracle/admin/ccdev1/bdump/ccdev1_smon_12610.trc:
ORA-00600: internal error code, arguments: [2662], [2223], [1767217797], [2223], [1767218866], [205521621], [], []
Mon Apr 26 20:40:19 2010
Errors in file /oracle/admin/ccdev1/bdump/ccdev1_smon_12610.trc:
ORA-00600: internal error code, arguments: [2662], [2223], [1767217797], [2223], [1767218866], [205521621], [], []
Non-fatal internal error happenned while SMON was doing extent coalescing.
SMON encountered 2 out of maximum 100 non-fatal internal errors.
Mon Apr 26 20:40:21 2010
Errors in file /oracle/admin/ccdev1/bdump/ccdev1_smon_12610.trc:
ORA-00600: internal error code, arguments: [2662], [2223], [1767217798], [2223], [1767218866], [205521621], [], []
Non-fatal internal error happenned while SMON was doing extent coalescing.
SMON encountered 3 out of maximum 100 non-fatal internal errors.
Mon Apr 26 20:40:23 2010
Errors in file /oracle/admin/ccdev1/bdump/ccdev1_smon_12610.trc:
ORA-00600: internal error code, arguments: [2662], [2223], [1767217799], [2223], [1767218866], [205521621], [], []
Non-fatal internal error happenned while SMON was doing extent coalescing.
SMON encountered 4 out of maximum 100 non-fatal internal errors.
Mon Apr 26 20:40:34 2010
Errors in file /oracle/admin/ccdev1/bdump/ccdev1_smon_12610.trc:
ORA-00600: internal error code, arguments: [2662], [2223], [1767217803], [2223], [1767218866], [205521621], [], []
Mon Apr 26 20:40:35 2010
Non-fatal internal error happenned while SMON was doing extent coalescing.
SMON encountered 5 out of maximum 100 non-fatal internal errors.
Mon Apr 26 20:40:45 2010
Errors in file /oracle/admin/ccdev1/bdump/ccdev1_smon_12610.trc:
ORA-00600: internal error code, arguments: [2662], [2223], [1767217806], [2223], [1767218866], [205521621], [], []
Mon Apr 26 20:40:47 2010
Non-fatal internal error happenned while SMON was doing extent coalescing.
SMON encountered 6 out of maximum 100 non-fatal internal errors.
Mon Apr 26 20:40:57 2010
Errors in file /oracle/admin/ccdev1/bdump/ccdev1_smon_12610.trc:
ORA-00600: internal error code, arguments: [2662], [2223], [1767217810], [2223], [1767218866], [205521621], [], []
Mon Apr 26 20:40:58 2010
Non-fatal internal error happenned while SMON was doing extent coalescing.
SMON encountered 7 out of maximum 100 non-fatal internal errors.
Mon Apr 26 20:41:08 2010
Errors in file /oracle/admin/ccdev1/bdump/ccdev1_smon_12610.trc:
ORA-00600: internal error code, arguments: [2662], [2223], [1767217814], [2223], [1767218866], [205521621], [], []
Mon Apr 26 20:41:10 2010
Errors in file /oracle/admin/ccdev1/bdump/ccdev1_pmon_12592.trc:
ORA-00474: SMON process terminated with error
Mon Apr 26 20:41:10 2010
PMON: terminating instance due to error 474
Instance terminated by PMON, pid = 12592

||||||||||||||||||||||||||||||||||||||||||||||||||
这是一次数据库的完整的启动过程，
启动之后 数据库的报错信息如下所示
ORA-00600: internal error code, arguments: [2663], [2223], [1767217797], [2223], [1767218866], [], [], []
1767217797 是数据库的scn，
1767218866 是一个datafile的scn。
可以看到，数据库启动之后，smon将做Instance recovery操作，具体原因是数据库的database 将应用联机日志做前滚操作。
当滚到一定的时间点之后，还是没有能把数据库Open，
smon 终止并报错误
ORA-00474: SMON process terminated with error
最后pmon发现smon不存在，将把instance 终止
PMON: terminating instance due to error 474
Instance terminated by PMON, pid = 12592
   

oracle工程师分析故障原因，数据库正常运行的时候，因为磁盘阵列的故障，造成了IO丢失，这个时候，dbwr写到数据库中的
一个datafile的io顺利执行了，但是同时去跟新ctlfile的io丢失了。造成一个问题，即数据库datafile的scn(1767218866)比数据库时间(1767217797)要更新。
这样每次重启的时候，数据库的库scn将通过redo log前滚（instance recovery),可以看到因为库的scn一直是在向前追的过程中，
oracle工程师判断再重启几次之后，数据库的库时间将追上这个datafile的时间。
下面显示数据库时间（蓝色字体），datafile时间（紫色字体）

1767217797
1767217798
1767217799
1767217803
1767217806
1767217810
1767217814
1767218866

从上面可以看出，数据库的scn是一直向前追的。

果然再次启动数据库之后，2663 的600号错误小时，换成了4194的 600号错误，这个和回滚段的故障有关。
错误信息如下所示：
Mon Apr 26 20:42:05 2010
Errors in file /oracle/admin/ccdev1/bdump/ccdev1_smon_12753.trc:
ORA-00600: internal error code, arguments: [4194], [21], [8], [], [], [], [], []


下一阶段的错误信息如下所示：

||||||||||||||||||||||||||||||||||||||||||||
Mon Apr 26 20:55:15 2010
Starting ORACLE instance (normal)
LICENSE_MAX_SESSION = 0
LICENSE_SESSIONS_WARNING = 0
Picked latch-free SCN scheme 3
Using LOG_ARCHIVE_DEST_1 parameter default value as /oracle/product/10.2.0/db/dbs/arch
Autotune of undo retention is turned on.
IMODE=BR
ILAT =121
LICENSE_MAX_USERS = 0
SYS auditing is disabled
ksdpec: called for event 13740 prior to event group initialization
Starting up ORACLE RDBMS Version: 10.2.0.4.0.
System parameters with non-default values:
  processes                = 1000
  sessions                 = 1105
  event                    = 10015 trace name context forever, level 10
  sga_max_size             = 12884901888
  __shared_pool_size       = 5066719232
  __large_pool_size        = 16777216
  __java_pool_size         = 16777216
  __streams_pool_size      = 0
  sga_target               = 12884901888
  control_files            = /oradata/ccdev1/control01.ctl, /oradata/ccdev1/control02.ctl, /oradata/ccdev1/control03.ctl
  db_block_size            = 8192
  __db_cache_size          = 7767851008
  db_writer_processes      = 4
  compatible               = 10.2.0.3.0
  db_files                 = 300
  db_file_multiblock_read_count= 16
  undo_management          = AUTO
  undo_tablespace          = UNDOTBS1
  remote_login_passwordfile= EXCLUSIVE
  db_domain                =
  dispatchers              = (PROTOCOL=TCP) (SERVICE=ccdev1XDB)
  job_queue_processes      = 10
  background_dump_dest     = /oracle/admin/ccdev1/bdump
  user_dump_dest           = /oracle/admin/ccdev1/udump
  core_dump_dest           = /oracle/admin/ccdev1/cdump
  audit_file_dest          = /oracle/admin/ccdev1/adump
  db_name                  = ccdev1
  open_cursors             = 300
  pga_aggregate_target     = 4294967296
PMON started with pid=2, OS id=13592
PSP0 started with pid=3, OS id=13594
MMAN started with pid=4, OS id=13596
DBW0 started with pid=5, OS id=13598
DBW1 started with pid=6, OS id=13600
DBW2 started with pid=7, OS id=13602
DBW3 started with pid=8, OS id=13604
LGWR started with pid=9, OS id=13606
CKPT started with pid=10, OS id=13608
SMON started with pid=11, OS id=13610
RECO started with pid=12, OS id=13612
CJQ0 started with pid=13, OS id=13614
MMON started with pid=14, OS id=13616
Mon Apr 26 20:55:17 2010
starting up 1 dispatcher(s) for network address '(ADDRESS=(PARTIAL=YES)(PROTOCOL=TCP))'...
MMNL started with pid=15, OS id=13618
Mon Apr 26 20:55:17 2010
starting up 1 shared server(s) ...
Mon Apr 26 20:55:17 2010
ALTER DATABASE   MOUNT
Mon Apr 26 20:55:24 2010
Setting recovery target incarnation to 2
Mon Apr 26 20:55:24 2010
Successful mount of redo thread 1, with mount id 1621322232
Mon Apr 26 20:55:24 2010
Database mounted in Exclusive Mode
Completed: ALTER DATABASE   MOUNT
Mon Apr 26 20:55:24 2010
ALTER DATABASE OPEN
Mon Apr 26 20:55:24 2010
Beginning crash recovery of 1 threads
 parallel recovery started with 3 processes
Mon Apr 26 20:55:24 2010
Started redo scan
Mon Apr 26 20:55:25 2010
Completed redo scan
 2 redo blocks read, 1 data blocks need recovery
Mon Apr 26 20:55:25 2010
Started redo application at
 Thread 1: logseq 1342, block 55
Mon Apr 26 20:55:25 2010
Recovery of Online Redo Log: Thread 1 Group 1 Seq 1342 Reading mem 0
  Mem# 0: /oradata/ccdev1/redo01.log
Mon Apr 26 20:55:25 2010
Completed redo application
Mon Apr 26 20:55:25 2010
Completed crash recovery at
 Thread 1: logseq 1342, block 57, scn 9549479558100
 1 data blocks read, 1 data blocks written, 2 redo blocks read
Mon Apr 26 20:55:25 2010
Thread 1 advanced to log sequence 1343 (thread open)
Thread 1 opened at log sequence 1343
  Current log# 2 seq# 1343 mem# 0: /oradata/ccdev1/redo02.log
Successful open of redo thread 1
Mon Apr 26 20:55:25 2010
MTTR advisory is disabled because FAST_START_MTTR_TARGET is not set
Mon Apr 26 20:55:25 2010
SMON: enabling cache recovery
Mon Apr 26 20:55:25 2010
Successfully onlined Undo Tablespace 1.
Mon Apr 26 20:55:25 2010
SMON: enabling tx recovery
Mon Apr 26 20:55:25 2010
Database Characterset is ZHS16GBK
Opening with internal Resource Manager plan
where NUMA PG = 1, CPUs = 4
replication_dependency_tracking turned off (no async multimaster replication found)
Starting background process QMNC
QMNC started with pid=22, OS id=13635
Mon Apr 26 20:55:27 2010
Errors in file /oracle/admin/ccdev1/udump/ccdev1_ora_13624.trc:
ORA-00600: internal error code, arguments: [4194], [45], [4], [], [], [], [], []
Mon Apr 26 20:55:28 2010
Errors in file /oracle/admin/ccdev1/bdump/ccdev1_smon_13610.trc:
ORA-00600: internal error code, arguments: [4194], [21], [8], [], [], [], [], []
Mon Apr 26 20:55:30 2010
Doing block recovery for file 49 block 725
Block recovery from logseq 1343, block 42 to scn 9549479558193
Mon Apr 26 20:55:30 2010
Recovery of Online Redo Log: Thread 1 Group 2 Seq 1343 Reading mem 0
  Mem# 0: /oradata/ccdev1/redo02.log
Block recovery stopped at EOT rba 1343.43.16
Block recovery completed at rba 1343.43.16, scn 2223.1767259183
Doing block recovery for file 2 block 9
Block recovery from logseq 1343, block 42 to scn 9549479558190
Mon Apr 26 20:55:30 2010
Recovery of Online Redo Log: Thread 1 Group 2 Seq 1343 Reading mem 0
  Mem# 0: /oradata/ccdev1/redo02.log
Block recovery completed at rba 1343.43.16, scn 2223.1767259183
Mon Apr 26 20:55:30 2010
Errors in file /oracle/admin/ccdev1/bdump/ccdev1_smon_13610.trc:
ORA-01595: error freeing extent (10) of rollback segment (1))
ORA-00607: Internal error occurred while making a change to a data block
ORA-00600: internal error code, arguments: [4194], [21], [8], [], [], [], [], []
Mon Apr 26 20:56:26 2010
Doing block recovery for file 50 block 5235
Doing block recovery for file 2 block 105
Block recovery from logseq 1343, block 41 to scn 9549479558189
Mon Apr 26 20:56:26 2010
Recovery of Online Redo Log: Thread 1 Group 2 Seq 1343 Reading mem 0
  Mem# 0: /oradata/ccdev1/redo02.log
Block recovery completed at rba 1343.42.16, scn 2223.1767259182
Doing block recovery for file 50 block 5235
Block recovery from logseq 1343, block 41 to scn 9549479558215
Mon Apr 26 20:56:26 2010
Recovery of Online Redo Log: Thread 1 Group 2 Seq 1343 Reading mem 0
  Mem# 0: /oradata/ccdev1/redo02.log
Block recovery stopped at EOT rba 1343.45.16
Block recovery completed at rba 1343.45.16, scn 2223.1767259191
Mon Apr 26 21:10:34 2010
Errors in file /oracle/admin/ccdev1/bdump/ccdev1_smon_13610.trc:
ORA-00600: internal error code, arguments: [4194], [44], [9], [], [], [], [], []
Mon Apr 26 21:10:37 2010
Doing block recovery for file 49 block 47529
Block recovery from logseq 1343, block 51 to scn 9549479564499
Mon Apr 26 21:10:37 2010
Recovery of Online Redo Log: Thread 1 Group 2 Seq 1343 Reading mem 0
  Mem# 0: /oradata/ccdev1/redo02.log
Block recovery stopped at EOT rba 1343.52.16
Block recovery completed at rba 1343.52.16, scn 2223.1767265490
Doing block recovery for file 2 block 153
Block recovery from logseq 1343, block 51 to scn 9549479564497
Mon Apr 26 21:10:37 2010
Recovery of Online Redo Log: Thread 1 Group 2 Seq 1343 Reading mem 0
  Mem# 0: /oradata/ccdev1/redo02.log
Block recovery completed at rba 1343.52.16, scn 2223.1767265490
Mon Apr 26 21:10:37 2010
Errors in file /oracle/admin/ccdev1/bdump/ccdev1_smon_13610.trc:
ORA-00607: Internal error occurred while making a change to a data block
ORA-00600: internal error code, arguments: [4194], [44], [9], [], [], [], [], []
Mon Apr 26 21:10:38 2010
Errors in file /oracle/admin/ccdev1/bdump/ccdev1_smon_13610.trc:
ORA-00600: internal error code, arguments: [4194], [57], [26], [], [], [], [], []
Doing block recovery for file 50 block 184225
Block recovery from logseq 1343, block 52 to scn 9549479564503
Mon Apr 26 21:10:41 2010
Recovery of Online Redo Log: Thread 1 Group 2 Seq 1343 Reading mem 0
  Mem# 0: /oradata/ccdev1/redo02.log
Block recovery stopped at EOT rba 1343.53.16
Block recovery completed at rba 1343.53.16, scn 2223.1767265494
Doing block recovery for file 2 block 73
Block recovery from logseq 1343, block 52 to scn 9549479564501
Mon Apr 26 21:10:41 2010
Recovery of Online Redo Log: Thread 1 Group 2 Seq 1343 Reading mem 0
  Mem# 0: /oradata/ccdev1/redo02.log
Block recovery completed at rba 1343.53.16, scn 2223.1767265494
Mon Apr 26 21:10:41 2010
Errors in file /oracle/admin/ccdev1/bdump/ccdev1_smon_13610.trc:
ORA-00607: Internal error occurred while making a change to a data block
ORA-00600: internal error code, arguments: [4194], [57], [26], [], [], [], [], []
Mon Apr 26 21:10:42 2010
Errors in file /oracle/admin/ccdev1/bdump/ccdev1_smon_13610.trc:
ORA-00600: internal error code, arguments: [4194], [59], [14], [], [], [], [], []
Doing block recovery for file 50 block 2692
Block recovery from logseq 1343, block 53 to scn 9549479564506
Mon Apr 26 21:10:45 2010
Recovery of Online Redo Log: Thread 1 Group 2 Seq 1343 Reading mem 0
  Mem# 0: /oradata/ccdev1/redo02.log
Block recovery stopped at EOT rba 1343.54.16
Block recovery completed at rba 1343.54.16, scn 2223.1767265497
Doing block recovery for file 2 block 89
Block recovery from logseq 1343, block 53 to scn 9549479564504
Mon Apr 26 21:10:45 2010
Recovery of Online Redo Log: Thread 1 Group 2 Seq 1343 Reading mem 0
  Mem# 0: /oradata/ccdev1/redo02.log
Block recovery completed at rba 1343.54.16, scn 2223.1767265497
Mon Apr 26 21:10:45 2010
Errors in file /oracle/admin/ccdev1/bdump/ccdev1_smon_13610.trc:
ORA-00607: Internal error occurred while making a change to a data block
ORA-00600: internal error code, arguments: [4194], [59], [14], [], [], [], [], []
Mon Apr 26 21:10:46 2010
Errors in file /oracle/admin/ccdev1/bdump/ccdev1_smon_13610.trc:
ORA-00600: internal error code, arguments: [4194], [6], [3], [], [], [], [], []
Doing block recovery for file 50 block 1405
Block recovery from logseq 1343, block 54 to scn 9549479564509
Mon Apr 26 21:10:49 2010
Recovery of Online Redo Log: Thread 1 Group 2 Seq 1343 Reading mem 0
  Mem# 0: /oradata/ccdev1/redo02.log
Block recovery stopped at EOT rba 1343.55.16
Block recovery completed at rba 1343.55.16, scn 2223.1767265500
Doing block recovery for file 2 block 41
Block recovery from logseq 1343, block 54 to scn 9549479564507
Mon Apr 26 21:10:49 2010
Recovery of Online Redo Log: Thread 1 Group 2 Seq 1343 Reading mem 0
  Mem# 0: /oradata/ccdev1/redo02.log
Block recovery completed at rba 1343.55.16, scn 2223.1767265500
Mon Apr 26 21:10:49 2010
Errors in file /oracle/admin/ccdev1/bdump/ccdev1_smon_13610.trc:
ORA-00607: Internal error occurred while making a change to a data block
ORA-00600: internal error code, arguments: [4194], [6], [3], [], [], [], [], []
Mon Apr 26 21:10:50 2010
Errors in file /oracle/admin/ccdev1/bdump/ccdev1_smon_13610.trc:
ORA-00600: internal error code, arguments: [4194], [44], [9], [], [], [], [], []
Doing block recovery for file 49 block 47529
Block recovery from logseq 1343, block 51 to scn 9549479564499
Mon Apr 26 21:10:53 2010
Recovery of Online Redo Log: Thread 1 Group 2 Seq 1343 Reading mem 0
  Mem# 0: /oradata/ccdev1/redo02.log
Block recovery completed at rba 1343.52.16, scn 2223.1767265493
Doing block recovery for file 2 block 153
Block recovery from logseq 1343, block 51 to scn 9549479564513
Mon Apr 26 21:10:53 2010
Recovery of Online Redo Log: Thread 1 Group 2 Seq 1343 Reading mem 0
  Mem# 0: /oradata/ccdev1/redo02.log
Block recovery completed at rba 1343.57.16, scn 2223.1767265506
Mon Apr 26 21:10:53 2010
Errors in file /oracle/admin/ccdev1/bdump/ccdev1_smon_13610.trc:
ORA-00607: Internal error occurred while making a change to a data block
ORA-00600: internal error code, arguments: [4194], [44], [9], [], [], [], [], []
Mon Apr 26 21:11:03 2010
Errors in file /oracle/admin/ccdev1/bdump/ccdev1_smon_13610.trc:
ORA-00600: internal error code, arguments: [4194], [57], [26], [], [], [], [], []
Mon Apr 26 21:11:06 2010
Doing block recovery for file 50 block 184225
Block recovery from logseq 1343, block 52 to scn 9549479564503
Mon Apr 26 21:11:06 2010
Recovery of Online Redo Log: Thread 1 Group 2 Seq 1343 Reading mem 0
  Mem# 0: /oradata/ccdev1/redo02.log
Block recovery completed at rba 1343.53.16, scn 2223.1767265496
Doing block recovery for file 2 block 73
Block recovery from logseq 1343, block 52 to scn 9549479564518
Mon Apr 26 21:11:06 2010
Recovery of Online Redo Log: Thread 1 Group 2 Seq 1343 Reading mem 0
  Mem# 0: /oradata/ccdev1/redo02.log
Block recovery completed at rba 1343.58.16, scn 2223.1767265511
Mon Apr 26 21:11:06 2010
Errors in file /oracle/admin/ccdev1/bdump/ccdev1_smon_13610.trc:
ORA-00607: Internal error occurred while making a change to a data block
ORA-00600: internal error code, arguments: [4194], [57], [26], [], [], [], [], []
Mon Apr 26 21:11:16 2010
Errors in file /oracle/admin/ccdev1/bdump/ccdev1_smon_13610.trc:
ORA-00600: internal error code, arguments: [4194], [21], [8], [], [], [], [], []
Mon Apr 26 21:11:19 2010
Doing block recovery for file 49 block 725
Block recovery from logseq 1343, block 58 to scn 9549479564526
Mon Apr 26 21:11:19 2010
Recovery of Online Redo Log: Thread 1 Group 2 Seq 1343 Reading mem 0
  Mem# 0: /oradata/ccdev1/redo02.log
Block recovery stopped at EOT rba 1343.61.16
Block recovery completed at rba 1343.61.16, scn 2223.1767265517
Doing block recovery for file 2 block 9
Block recovery from logseq 1343, block 58 to scn 9549479564524
Mon Apr 26 21:11:19 2010
Recovery of Online Redo Log: Thread 1 Group 2 Seq 1343 Reading mem 0
  Mem# 0: /oradata/ccdev1/redo02.log
Block recovery completed at rba 1343.61.16, scn 2223.1767265517
Mon Apr 26 21:11:19 2010
Errors in file /oracle/admin/ccdev1/bdump/ccdev1_smon_13610.trc:
ORA-00607: Internal error occurred while making a change to a data block
ORA-00600: internal error code, arguments: [4194], [21], [8], [], [], [], [], []
Mon Apr 26 21:11:29 2010
Errors in file /oracle/admin/ccdev1/bdump/ccdev1_smon_13610.trc:
ORA-00600: internal error code, arguments: [4194], [44], [9], [], [], [], [], []
Mon Apr 26 21:11:32 2010
Errors in file /oracle/admin/ccdev1/bdump/ccdev1_pmon_13592.trc:
ORA-00474: SMON process terminated with error
Mon Apr 26 21:11:32 2010
PMON: terminating instance due to error 474
Instance terminated by PMON, pid = 13592
 
我们查询了一下file 2和 file 50对应的 数据文件，都是Undo tablespace的datafile。
oracle 工程师解释 4194号错误是因为undo tablespace里面的坏块错误导致的数据库错误。
发生这个错误的时候，开始阶段数据库处于open状态，但是过了一会，smon因为发现了undo 的坏块将终止，
最后导致pmon 因为smon的丢失而把instance 干掉。

 

解决的方法分为两种，
查看数据库的Undo tablespace 里面是否有活动的会话。(通过10015事件）

1、如果没有，可以直接用新的Undo tablespace，然后drop掉这个老的undo tablespace
2、如果有，则需要做其他操作
查看dba_corruption_block 来找到corrupt的块

然后选择 corruption 方式来解决。

||||||||||||||||||||||||||||||||||||||||||||||||||||||

在数据库层面设置10015事件，
events = "10015 trace name context forever,level 10" scope=spfile
再重启数据库，
检查�

傻儿哥
关注
专栏目录
【Java从入门到天黑|01】SE入门之Java的前生今世
孙叫兽的博客
04-16 3846
JavaSE入门基础
SQL Server专家的10个秘诀（翻译加注解）
热门推荐
MVP黄钊吉（發糞塗牆）
02-14 1万+
当你点开这篇文章的时候，如果觉得没有读下去的必要，也希望你能拉到最后看看那几行字！原文出处：https://technet.microsoft.com/en-us/magazine/gg299551.aspx译者注： 本文主要以翻译为主，但由于工作经历，译者也有一些自己的看法和心得，并且不打算完全照搬直译，期间过滤了一些本人觉得无关紧要的信息，如果读者觉得看的别扭，可以看原文。 这篇文章可能已被广
oracle 技术支持 生命周期,Oracle Lifetime Support(支持生命周期) 说明
weixin_39737636的博客
04-03 459
一.Oracle Lifetime Support 分类网站的说明在如下这篇文章里有更详细的说明：Oracle Lifetime Support PolicyWith OracleSupport, you know up front and with certainty how long your Oracle products aresupported. The Lifetime Support...
oracle support网站,Oracle Support 站点弃Flash转HTML重建
weixin_39839968的博客
04-09 260
Oracle Support 站点弃Flash转HTML重建尊敬的 My Oracle Support 用户,2012年1月，我们推出了一个新的基于HTML的My Oracle Support用户界面。新的用户界面使 用Oracle应用开发框架，是我们对客户和合作伙伴提供一个单一的在线支持门户的第一步，在未 来几个月内所有用户都将过渡到这个界面。为了有助于平稳过渡，我们邀请您预览并开始使用新的My...
Oracle中joint,oracle support
weixin_34670892的博客
04-06 330
什么是Joint Escalation Team?Joint Escalation Team也可以简写为JET。JET是指当客户采用了多种软件供应商的产品搭建系统后，系统所发生的问题通过单独的某一个供应商无法解决，需要集合多个供应商的技术力量解决，这个时候就需要组成Joint Escalation Team，从而避免陷入孤军奋战。Joint Escalation Team可以由客户自己牵头，也可以...
oracle support一次危机处理（之一）
04-28 3424
天有不测风云，IBM 的中端磁盘阵列这次又惹祸了。在微码升级的时候，DS4800 发生故障。IBM工程师把阵列故障恢复后，拍拍屁股走人，留下了阵列上不一致的数据。没办法，阵列上的数据又得我们来想办法抢救了。。。。还好有oracle support的支持我们最终化解了这次危机。下文详细介绍了整个恢复的过程。  oracle工程师到达现场后，根据错误信息， 问题情况
MOS（My Oracle Support）怎么用？Oracle DBA必备技能！
最新发布
xiaofan23z的专栏
04-29 1495
大中华区开立的SR 其实是可以使用中文提交的，level3-4的SR一般都是Oracle 大连技术支持中心在支持，如果是level1-2需要24小时持续工作的，非大陆工作时间会将SR转给其他时区的技术支持中心比如印度，欧洲美洲等，这就需要英文来交流！默认提SR都需要按oracle 官方的TFA收集信息上传 以供分析，我这里是sql性能问题 ，我提供的部分执行计划，sqlhc，awr等信息。------------历史文章推荐---------------
Scratch 被禁止访问；Linux 内核历史报告2020 等开源之道每周评论2020 09 15
johnwoo_lee的专栏
09-15 1072
点击上方蓝字，关注开源之道声明：本文所言论，仅代表适兕个人观点论文阅读心得与体会Linux 内核历史报告2020原文链接：the 2020 Linux Kernel History Re...
基于Apriori算法的高职大学生就业能力的研究
zxg_1991的专栏
03-08 6520
引言 最近在写关于Apriori算法的研究论文，这一部分分享出来给大家研究。 摘要 近几年来,随着高校扩招规模不断地扩大,每年都有大量的应届毕业生面临就业问题,高校就业指导工作就显得尤为重要。对于高校而言,毕业生的就业率成为评估高校办学水平和学校社会声誉的重要指标,就业率低下将成为影响高校生存发展的制约因素。在高校就业指导部门的就业管理信息系统中积累了大量的、重要的学生就业信息资源，虽然大多数就业...
【JVM系列4】JVM常用命令
飞楼踏梦
08-19 288
讲解JVM的常用命令，为后面讲解JVM日志分析做准备。 前言 JVM比较重要的知识，在前面三篇文章都将完了，GC日志在第一篇文章中也有所提及，这篇文章主要讲解JVM常用的命令，虽然jvm调优成熟的工具已经有很多，但是所有的工具几乎都是依赖于jdk的接口和底层的这些命令，研究这些命令的使用也让我们更能了解jvm构成和特性。 Sun JDK监控和故障处理命令有jps、jstat、jmap、jhat、jstack、jinfo，下面做一一介绍。 jps JVM Process Status To...
看看Oracle的support是如何打太极的
悠悠红叶
10-29 1497
当然，也许是水平有限的原因前几天，我自己的数据库突然出了问题，我的scott用户没有建表的权限,后来发现其实所有的普通用户都没有DDL/DML权限，一旦执行就报1031错误。我自己确实没有做过什么操作的，我就想看看Oracle的高人们会怎么解决，当然，由于是我自己的系统，我就先建了一个在用，等着Oracle的给我support，因为很早就知道Oracle的人打太极很厉害。25日申请了ITAR，Mu
Terminated With Error ORA-474: SMON Process Terminated With Error-1361872.1
01-22 771
Instance Terminated With Error ORA-00474: SMON Process Terminated With Error [ID 1361872.1]Applies to: ...
一次BT的故障处理
03-21 617
记录一次在我看来比较另类的故障处理。环境：oracle 10.2.0.3，OS:windows 32bit。现象：数据库startup后，10-30秒之内出现崩溃，自动关闭。alert中的报错如下...
ORA-00600 [13013] [5001] [474] [4198011] [102]-Oracle 问题处理过程
12-27 632
ORA-00600 [13013] [5001] [474] [4198011] [102] [4198011] [17]-记录一次Oracle ORA-00600 [13013]处理过程 redhat linux 4.7 平...
Oracle共享账户
weixin_58818322的博客
08-09 920
http://bugmenot.com/view/oracle.com
ORA-00600: 内部错误代码, 参数: [13013]
ckth47350的博客
06-11 2081
ORA-00600: 内部错误代码, 参数: [13013] 数据库版本：Oracle 11.2.0.1.0 数据库服务器操作系统：Windows server 20...
07 oracle 归档模式 inactive/current redo log损坏修复--以及错误ORA-00600: internal error code, arguments: [2663],...
weixin_30421809的博客
06-06 354
07 oracle 归档模式 inactive/current redo log损坏修复--以及错误ORA-00600: internal error code, arguments: [2663], [0], [9710724], [0], [9711142], [], [], [], [], [], [], [] --环境： SQL*Plus: Release 11.2.0.4.0 Pro...
TTS（表空间迁移）遇到的BUG
asd3262584的专栏
03-29 921
  最近从IBM Power 平台迁移数据到LINUX平台，使用TTS进行迁移，Power平台原始数据库中有3张大数据量的分区表，无法使用TTS进行迁移，查资料没找到好的方法来对分区表做跨平台迁移，于是才用Spool导出数据->删除分区表->表空间迁移->新建表->sqlldr导入数据的办法。前面一切都正常，到最后sqlldr导入数据库时，数据突然崩溃掉，查看ALTER 日志，没看出是什么引起
Oracle VM VirtualBox二次开发指南
"Oracle VM VirtualBox的二次开发文档，涵盖了从4.1.0版本的编程指南和参考内容，包括VirtualBox的模块化结构、Web服务接口、多种语言的使用方法以及COM/XPCOM直接调用等技术细节。" 本文档详细介绍了VirtualBox的...
评论 1
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值