Oracle Database Discussions

I had issues with UNDO. Here is what happened -

SQL> alter tablespace UNDO
2 add datafile '/oracle/TESTDB/data04/TESTDB_undo_04.db' size 100M autoextend on maxsize 10000M;

SQL> select file_name,bytes/1024/1024 MB, maxbytes/1024/1024 MMB from dba_data_files where tablespace_name='UNDO';

FILE_NAME MB MMB
------------------------------------------------------- ---------- ----------
/oracle/TESTDB/data02/TESTDB_undo_01.dbf 6158 10240
/oracle/TESTDB/data01/TESTDB_undo_02.dbf 7703 0
/oracle/TESTDB/data03/TESTDB_undo_03.dbf 7636 0
/oracle/TESTDB/data04/TESTDB_undo_04.db 100 10000

But after some time this same query gave me this error -

SQL> select file_name,bytes/1024/1024 MB, maxbytes/1024/1024 MMB from dba_data_files where tablespace_name='UNDO';

FILE_NAME MB MMB
------------------------------------------------------- ---------- ----------
/oracle/TESTDB/data02/TESTDB_undo_01.dbf 6158 10240
/oracle/TESTDB/data01/TESTDB_undo_02.dbf 7703 0
/oracle/TESTDB/data03/TESTDB_undo_03.dbf 7636 0
/oracle/TESTDB/data04/TESTDB_undo_04.db

Users then started complaining about it.

Queries like this also started giving us these problems -

SQL> select file_id,file_name,bytes/1024/1024 MB, maxbytes/1024/1024 MMB,status from dba_data_files where tablespace_name='UNDO';
select file_id,file_name,bytes/1024/1024 MB, maxbytes/1024/1024 MMB,status from dba_data_files where tablespace_name='UNDO'
*
ERROR at line 1:
ORA-00376: file 67 cannot be read at this time
ORA-01110: data file 67: '/oracle/TESTDB/data04/TESTDB_undo_04.db'


Then I though of bouncing the database but then it also giving some problem -

SQL> shutdown immediate;
ORA-00376: file 67 cannot be read at this time
ORA-01110: data file 67: '/oracle/TESTDB/data04/TESTDB_undo_04.db'


SQL> exit
Disconnected from Oracle Database 11g Enterprise Edition Release 11.1.0.7.0 - 64bit Production
With the Partitioning, OLAP, Data Mining and Real Application Testing options
TESTDB - /oracle/TESTDB/data03> sqlplus / as sysdba

SQL*Plus: Release 11.1.0.7.0 - Production on Fri Jul 30 09:27:24 2010

Copyright (c) 1982, 2008, Oracle. All rights reserved.

Connected.
SQL> shutdown abort;
ORACLE instance shut down.
SQL> startup;
ORACLE instance started.

Total System Global Area 4175568896 bytes
Fixed Size 2160352 bytes
Variable Size 3137341728 bytes
Database Buffers 1023410176 bytes
Redo Buffers 12656640 bytes
Database mounted.
ORA-00376: file 67 cannot be read at this time
ORA-01110: data file 67: '/oracle/TESTDB/data04/TESTDB_undo_04.db'



Thought of recovering the this tablespace but still getting this error -

RMAN> recover tablespace "UNDO";

Starting recover at 30-JUL-10
starting full resync of recovery catalog
full resync complete
allocated channel: ORA_SBT_TAPE_1
channel ORA_SBT_TAPE_1: SID=278 device type=SBT_TAPE
channel ORA_SBT_TAPE_1: Data Protection for Oracle: version 5.5.1.0
allocated channel: ORA_DISK_1
channel ORA_DISK_1: SID=277 device type=DISK
RMAN-00571: ===========================================================
RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============
RMAN-00571: ===========================================================
RMAN-03002: failure of recover command at 07/30/2010 09:34:17
ORA-00376: file 67 cannot be read at this time
ORA-01110: data file 67: '/oracle/TESTDB/data04/TESTDB_undo_04.db'


Will greatly appreciate if someone can give some advice here.