突然收到备库上以下错误,吓了一大跳,hehe
根盘空间有3多个G呀,怎么会突然暴满?
哎,别想了,先解决问题要紧
Jul 28 16:20:13 hpdb2 vmunix: vxfs: NOTICE: msgcnt 1895 mesg 001: V-2-1: vx_nospace - /dev/root file system full (1 block ext
ent)
Jul 28 16:20:28 hpdb2 vmunix: vxfs: NOTICE: msgcnt 1897 mesg 001: V-2-1: vx_nospace - /dev/root file system full (1 block ext
ent)
Jul 28 16:20:56 hpdb2 vmunix: vxfs: NOTICE: msgcnt 1899 mesg 001: V-2-1: vx_nospace - /dev/root file system full (1 block ext
ent)
Jul 28 16:21:15 hpdb2 vmunix: vxfs: NOTICE: msgcnt 1901 mesg 001: V-2-1: vx_nospace - /dev/root file system full (1 block ext
ent)
Jul 28 16:21:30 hpdb2 vmunix: vxfs: NOTICE: msgcnt 1903 mesg 001: V-2-1: vx_nospace - /dev/root file system full (1 block ext
ent)
bdf确认了一下,根盘使用100%了.汗一个。
root@hpdb2:/jbindx02/oraindx/indx #bdf
Filesystem kbytes used avail %used Mounted on
/dev/vg00/lvol3 3145728 3145728 0 100% /
/dev/vg00/lvol1 505392 129008 325840 28% /stand
/dev/vg00/lvol8 5242880 1534784 3679208 29% /var
/dev/vg00/lvol7 5242880 2020824 3196896 39% /usr
/dev/vg00/u01 14680064 8635816 5998560 59% /u01
/dev/vg00/lvol6 2097152 1050976 1038128 50% /tmp
/dev/vg00/lvol5 5996544 3900936 2079272 65% /opt
/dev/vg00/lvol4 524288 83336 437552 16% /home
/dev/vgdgehrdata/lvolvgdgehrdata
819200000 435723320 380480784 53% /dgehrdata
/dev/vgdgehrarch/lvolvgdgehrarch
256000000 165329936 89961952 65% /dgehrarch
/dev/vgdba01/lvoldba01
1331232768 8767432 1312133584 1% /dba01
/dev/vgdba02/lvoldba02
1331232768 61856 1320771152 0% /dba02
/dev/vgdgjbarch/lvolvgdgjbarch
363528192 112804792 248764768 31% /dgjbarch
/dev/vgdgjbdata/lvolvgdgjbdata
1873936384 1122876504 745192272 60% /dgjbdata
糊乱找了一通,终于发现以下目录下多出了两个oracle的数据文件.
root@hpdb2:/jbindx02/oraindx/indx #ll
total 5596240
-rw-r----- 1 oracle dba 2097160192 Jul 28 09:24 rsm_idx80.dbf
-rw-r----- 1 oracle dba 771751936 Jul 28 09:24 rsm_idx81.dbf
嘿嘿,发现了就赶快删除吧,别急哦,不能删除? 为什么呢,请听我慢慢道来.(这里有点绕)
先说明介绍一下系统架构.
这个出问题的主机(简称Q主机),这个主机既是其他三台主机的HA(MC/SG),又是这台三台主机上数据的standby(Oracle/DG).
上面各位看到的那个/jbindx02/oraindx/indx 目录,其实是给HA/Clustering用的,一个没有mount任何存储的目录。
而dataguard其实是用得另外一套目录,由于在一台主库上有一个存放索引的目录/jbindx02/oraindx/indx 一直没有存放数据文件进去,所以就忘记了加在 db_file_name_convert中,今天加表空间时,看到其他目录快满了,就向这个目录放数据文件了,所以就造成了以上问题。
问题找到了,想办法解决备库问题吧.
解决过程如下:
1.先查看一下alert.log
WARNING: File being created with same name as in Primary
Existing file may be overwritten
Mon Jul 28 09:24:10 2008
File #546 added to control file as 'UNNAMED00546'.
Originally created as:
'/jbindx02/oraindx/indx/rsm_idx81.dbf'
Recovery was unable to create the file as:
'/jbindx02/oraindx/indx/rsm_idx81.dbf'
Errors with log /dgjbarch/dgjb_1_27376_638807858.log
MRP0: Background Media Recovery terminated with error 19502
Mon Jul 28 09:24:10 2008
Errors in file /u01/oracle/admin/hpjob/bdump/dghpjob_mrp0_21025.trc:
ORA-19502: write error on file "/jbindx02/oraindx/indx/rsm_idx81.dbf", blockno 93696 (blocksize=8192)
ORA-27072: File I/O error
HP-UX Error: 2: No such file or directory
Additional information: 4
Additional information: 93696
Additional information: 540672
Mon Jul 28 09:24:51 2008
Some recovered datafiles maybe left media fuzzy
Media recovery may continue but open resetlogs may fail
Mon Jul 28 09:24:51 2008
Errors in file /u01/oracle/admin/hpjob/bdump/dghpjob_mrp0_21025.trc:
ORA-19502: write error on file "/jbindx02/oraindx/indx/rsm_idx81.dbf", blockno 93696 (blocksize=8192)
ORA-27072: File I/O error
HP-UX Error: 2: No such file or directory
Additional information: 4
Additional information: 93696
Additional information: 540672
Mon Jul 28 09:24:51 2008
MRP0: Background Media Recovery process shutdown (dghpjob)
2.先停止恢复
alter database recover managed standby database cancel ;
3.把需要转用的目录加进去
ALTER SYSTEM SET db_file_name_convert='xxx,xxx' scope=both;
4.关闭数据库
shtudown immediate;
5.移动数据文件
mv rsm_idx80.dbf /xxxx
mv rsm_idx81.dbf /xxxx
6.启动数据库到mount
ALTER SYSTEM SET standby_file_management='MANUAL' SCOPE=BOTH;
alter database rename file '/jbindx02/oraindx/indx/rsm_idx80.dbf' to '/dgjbdata/indx/rsm_idx80.dbf' ;
---成功
alter database rename file '/jbindx02/oraindx/indx/rsm_idx80.dbf' to '/dgjbdata/indx/rsm_idx80.dbf' ;
---失败(错误信息如下)
ORA-01111: name for data file 546 is unknown - rename to correct file
ORA-01110: data file 546: '/u01/oracle/product/10.2.0.3/dbs/UNNAMED00546'
ORA-01157: cannot identify/lock data file 546 - see DBWR trace file
ORA-01111: name for data file 546 is unknown - rename to correct file
ORA-01110: data file 546: '/u01/oracle/product/10.2.0.3/dbs/UNNAMED00546'
其实从上面看这个文件的大小和alert.log中的信息,已经知道了,这个文件没有创建成功。
7.对这个损坏的文件进行恢复
SQL> select name from v$datafile where file# = 546;
NAME
--------------------------------------------------------------------------------
/u01/oracle/product/10.2.0.3/dbs/UNNAMED00546
SQL> alter database create datafile '/u01/oracle/product/10.2.0.3/dbs/UNNAMED00546' as '/dgjbdata/indx/rsm_idx81.dbf';
Database altered.
SQL> select name from v$datafile where file# = 546;
NAME
--------------------------------------------------------------------------------
/dgjbdata/indx/rsm_idx81.dbf
-------过来了^_^
SQL> alter system set standby_file_management=AUTO scope=both;
SQL> alter database create datafile '/u01/oracle/product/10.2.0.3/dbs/UNNAMED00546' as '/dgjbdata/indx/rsm_idx81.dbf';
Database altered.
8.切换到恢复状态下
ALTER DATABASE RECOVER managed standby database disconnect from session
9.完工了,回家鸟 ^_^
来自 “ ITPUB博客 ” ,链接:http://blog.itpub.net/7364032/viewspace-414942/,如需转载,请注明出处,否则将追究法律责任。
转载于:http://blog.itpub.net/7364032/viewspace-414942/