Standby database problem
loboncMay 25 2009 — edited Jun 2 2009Hi everbody!
I have a database and just to be safe I have created a standby database for it. Unfortunately under the primary database I have run out of space (the new storage arrives soon), so I have changed my backup strategy: I have changed from RMAN backup to user managed backup. On the primary database I issue:
shutdown abort
startup
shutdown immediate
After I copy all the database files to a new location and after I startup the database again; everthing is working properly. But today morning my usual check (select name, applied from v$archived_log where applied='YES' and completion_time >= to_date('05/25/2009', 'mm/dd/yyyy');) on my standby database has given nothing as a result (on primary db there were several log swithces).
I have checked the alert.log:
***ALERT.LOG***
Sat May 23 20:15:31 2009
RFS[3]: Possible network disconnect with primary database
Redo Shipping Client Connected as PUBLIC
-- Connected User is Valid
RFS[4]: Assigned to RFS process 1432
RFS[4]: Identified database type as 'physical standby'
RFS[4]: No standby redo logfiles created
Sat May 23 22:16:25 2009
RFS[4]: Possible network disconnect with primary database
Closing latent archivelog for thread 1 sequence 37638
EOF located at block 61441 low SCN 0:-1320385275 next SCN 0:-1320385275
Latent archivelog 'O:\ARCHIVELOGS\UMFT\ARC37638_0651241795.001'
If you wish to failover to this standby database, you should use the
following command to manually register the archivelog for recovery:
ALTER DATABASE REGISTER LOGFILE 'O:\ARCHIVELOGS\UMFT\ARC37638_0651241795.001';
Sat May 23 22:16:29 2009
Fetching gap sequence in thread 1, gap sequence 37638-37638
FAL[client]: Error fetching gap sequence, no FAL server specified
Sat May 23 22:16:59 2009
FAL[client]: Failed to request gap sequence
GAP - thread 1 sequence 37638-37638
DBID 408075329 branch 651241795
FAL[client]: All defined FAL servers have been attempted.
-------------------------------------------------------------
Check that the CONTROL_FILE_RECORD_KEEP_TIME initialization
parameter is defined to a value that is sufficiently large
enough to maintain adequate log switch information to resolve
archivelog gaps.
-------------------------------------------------------------
Sat May 23 22:58:48 2009
Redo Shipping Client Connected as PUBLIC
-- Connected User is Valid
RFS[5]: Assigned to RFS process 3972
RFS[5]: Identified database type as 'physical standby'
Sat May 23 23:27:59 2009
RFS[5]: No standby redo logfiles created
Sat May 23 23:28:39 2009
RFS[5]: Archived Log: 'O:\ARCHIVELOGS\UMFT\ARC37639_0651241795.001'
Sat May 23 23:49:32 2009
RFS[5]: No standby redo logfiles created
Sat May 23 23:50:12 2009
***ALERT.LOG***
The logs are arriving to the standby database but non of them appling. I have checked the archive log named ARC37638_0651241795.001 and the size seemed different on the primary and the standby database, so I have copied it manually, and make a logswitch on the primary server, and issued the following command on the standby db:
ALTER DATABASE REGISTER LOGFILE 'O:\ARCHIVELOGS\UMFT\ARC37638_0651241795.001'
The result in alert.log:
***ALERT.LOG***
Mon May 25 11:12:57 2009
ALTER DATABASE REGISTER LOGFILE 'O:\ARCHIVELOGS\UMFT\ARC37638_0651241795.001'
Mon May 25 11:12:57 2009
There are 1 logfiles specified.
ALTER DATABASE REGISTER [PHYSICAL] LOGFILE
Register archivelog O:\ARCHIVELOGS\UMFT\ARC37638_0651241795.001 already exists
ORA-16089 signalled during: ALTER DATABASE REGISTER LOGFILE 'O:\ARCHIVELOGS\UMFT\ARC37638_0651241795.001'...
***ALERT.LOG***
The alert.log on standby db somewhere after the log switch on the primary db started to show the folling:
***ALERT.LOG***
Media Recovery Log O:\ARCHIVELOGS\UMFT\ARC37638_0651241795.001
Media Recovery Log O:\ARCHIVELOGS\UMFT\ARC37639_0651241795.001
Media Recovery Log O:\ARCHIVELOGS\UMFT\ARC37640_0651241795.001
Media Recovery Log O:\ARCHIVELOGS\UMFT\ARC37641_0651241795.001
Media Recovery Log O:\ARCHIVELOGS\UMFT\ARC37642_0651241795.001
... (there is a lot of similar lines here)
Mon May 25 11:15:20 2009
Media Recovery Waiting for thread 1 sequence 37787
Mon May 25 11:15:29 2009
RFS[5]: No standby redo logfiles created
RFS[5]: Archived Log: 'O:\ARCHIVELOGS\UMFT\ARC37787_0651241795.001'
Mon May 25 11:15:40 2009
Media Recovery Log O:\ARCHIVELOGS\UMFT\ARC37787_0651241795.001
Media Recovery Waiting for thread 1 sequence 37788
Mon May 25 11:25:28 2009
RFS[5]: No standby redo logfiles created
Mon May 25 11:26:23 2009
RFS[5]: Archived Log: 'O:\ARCHIVELOGS\UMFT\ARC37788_0651241795.001'
Mon May 25 11:26:25 2009
Media Recovery Log O:\ARCHIVELOGS\UMFT\ARC37788_0651241795.001
Media Recovery Waiting for thread 1 sequence 37789
***ALERT.LOG***
So it seemed to me that the manual copy and the log switch made the standby db working properly again, but if I issue my check again on my standby db the result:
***SQL***
SQL> select name, applied from v$archived_log where applied='YES' and completion_time >= to_date('05/23/2009','mm/dd/yyyy');
NAME APP
-------------------------------------------- ---
...... (there is a lot of similar lines here)
O:\ARCHIVELOGS\UMFT\ARC37630_0651241795.001 YES
O:\ARCHIVELOGS\UMFT\ARC37631_0651241795.001 YES
O:\ARCHIVELOGS\UMFT\ARC37632_0651241795.001 YES
O:\ARCHIVELOGS\UMFT\ARC37633_0651241795.001 YES
O:\ARCHIVELOGS\UMFT\ARC37634_0651241795.001 YES
O:\ARCHIVELOGS\UMFT\ARC37635_0651241795.001 YES
O:\ARCHIVELOGS\UMFT\ARC37636_0651241795.001 YES
O:\ARCHIVELOGS\UMFT\ARC37637_0651241795.001 YES !!!!!
O:\ARCHIVELOGS\UMFT\ARC37639_0651241795.001 YES !!!!!
O:\ARCHIVELOGS\UMFT\ARC37640_0651241795.001 YES
O:\ARCHIVELOGS\UMFT\ARC37641_0651241795.001 YES
O:\ARCHIVELOGS\UMFT\ARC37642_0651241795.001 YES
O:\ARCHIVELOGS\UMFT\ARC37643_0651241795.001 YES
O:\ARCHIVELOGS\UMFT\ARC37644_0651241795.001 YES
O:\ARCHIVELOGS\UMFT\ARC37645_0651241795.001 YES
O:\ARCHIVELOGS\UMFT\ARC37646_0651241795.001 YES
...... (there is a lot of similar lines here)
O:\ARCHIVELOGS\UMFT\ARC37784_0651241795.001 YES
O:\ARCHIVELOGS\UMFT\ARC37785_0651241795.001 YES
O:\ARCHIVELOGS\UMFT\ARC37638_0651241795.001 YES !!!!!
O:\ARCHIVELOGS\UMFT\ARC37786_0651241795.001 YES
O:\ARCHIVELOGS\UMFT\ARC37787_0651241795.001 YES
O:\ARCHIVELOGS\UMFT\ARC37788_0651241795.001 YES
O:\ARCHIVELOGS\UMFT\ARC37789_0651241795.001 YES
***SQL***
I have marked the lines I think show the error with several exclamation marks...
***SQL***
SQL> select max(sequence#) from v$log_history;
MAX(SEQUENCE#)
---------------
37791
***SQL***
The SQL above gives the same result on both primary and standby db. The logs are appling constantly, it seems working properly... But I think one logfile applied wrong...
Thanks for your reply,
Tamas Taller