RMAN backup with CA ArcServe exits with ORA-19502 and ORA-27030
703196Mar 25 2010 — edited Mar 26 2010Hi all,
we have a RAC database with the following characteristics
• DB - Oracle RAC (2 nodes Oracle Server Enterprise Edition 10.2.0.4);
• O.S. – Microsoft Windows Server 2003 R2 Standard x64 Edition;
• H.W. – Quad Core AMD Opteron 1.90 GHz, Ram 32 Gb.
We have ArcServe which uses RMAN for the bakcup on a phisical partition (a LUN actually).
2010-03-24 22:00:33. 87 [[ 5732]:( 5768)] ORACMD[INFO]before run command, Current user is Administrator
2010-03-24 22:00:33.868 [[ 5732]:( 2820)] ORACMD[ERR]ReadFile failed 109
2010-03-24 22:00:33.884 [[ 5732]:( 5768)] ORACMD[INFO]current file pointer is 0, size=110
2010-03-24 22:00:33.884 [[ 5732]:( 5768)] ORACMD[INFO]current file pointer is 110, size=110
2010-03-24 22:00:33.884 [[ 5732]:( 5768)] ORACMD[INFO]After run command, Current user is Administrator
2010-03-24 22:00:34. 71 [[ 5732]:( 5768)] ORACMD[INFO]before run command, Current user is Administrator
2010-03-24 22:00:34.774 [[ 5732]:( 6768)] ORACMD[ERR]ReadFile failed 109
2010-03-24 22:00:34.790 [[ 5732]:( 5768)] ORACMD[INFO]current file pointer is 0, size=110
2010-03-24 22:00:34.790 [[ 5732]:( 5768)] ORACMD[INFO]current file pointer is 110, size=110
2010-03-24 22:00:34.790 [[ 5732]:( 5768)] ORACMD[INFO]After run command, Current user is Administrator
2010-03-24 22:00:34.821 [[ 5732]:( 5768)] ORABL[INFO]szRMANCmd = c:\oracle\product\10.2.0\db_1\bin\rman.exe @C:\PROGRA~1\CA\ARCSER~2\X64\RMANSC~2\cat47.tmp
2010-03-24 22:00:34.821 [[ 5732]:( 5768)] ORACMD[INFO]before run command, Current user is Administrator
2010-03-24 22:00:34.821 [[ 5732]:( 5768)] ORACMD[ERR]ReleaseMutex ok
2010-03-24 23:35:51.693 [[ 5732]:( 5060)] ORACMD[ERR]ReadFile failed 109
2010-03-24 23:35:51.693 [[ 5732]:( 5768)] ORABL[ERR]execute RMAN command status 1
2010-03-24 23:35:51.709 [[ 5732]:( 5768)] ORACMD[INFO]current file pointer is 0, size=9150
2010-03-24 23:35:51.724 [[ 5732]:( 5768)] ORACMD[INFO]current file pointer is 2047, size=9150
2010-03-24 23:35:51.724 [[ 5732]:( 5768)] ORACMD[INFO]current file pointer is 4094, size=9150
2010-03-24 23:35:51.724 [[ 5732]:( 5768)] ORACMD[INFO]current file pointer is 6141, size=9150
2010-03-24 23:35:51.724 [[ 5732]:( 5768)] ORACMD[INFO]current file pointer is 8188, size=9150
2010-03-24 23:35:51.724 [[ 5732]:( 5768)] ORACMD[INFO]current file pointer is 9150, size=9150
2010-03-24 23:35:51.724 [[ 5732]:( 5768)] ORABL[ERR]output size=9150
2010-03-24 23:35:51.724 [[ 5732]:( 5768)] ORABL[ERR]Recovery Manager: Release 10.2.0.4.0 - Production on Wed Mar 24 22:00:34 2010
2010-03-24 23:35:51.724 [[ 5732]:( 5768)] ORABL[ERR]Copyright (c) 1982, 2007, Oracle. All rights reserved.
2010-03-24 23:35:51.724 [[ 5732]:( 5768)] ORABL[ERR]RMAN> CONNECT TARGET */PASSWORD(MASKED)
2010-03-24 23:35:51.724 [[ 5732]:( 5768)] ORABL[ERR]2> CONNECT CATALOG *
2010-03-24 23:35:51.724 [[ 5732]:( 5768)] ORABL[ERR]3> @C:\PROGRA~1\CA\ARCSER~2\X64\RMANSC~1\SA83C9~2.RMA
2010-03-24 23:35:51.724 [[ 5732]:( 5768)] ORABL[ERR]4> RUN{
2010-03-24 23:35:51.724 [[ 5732]:( 5768)] ORABL[ERR]5> ALLOCATE CHANNEL 'cccora01' DEVICE TYPE sbt FORMAT '_8edd9910-cfe5-45e3-a7a2-fc1939dd152f_%u_%p_%c_' PARMS=' ENV=(CA_BAB_MASTER_JOBID=27,CA_BAB_DUMMY_JOBID=2,CA_BAB_PROCESS_ID=2027284384,CA_BAB_THREAD_ID=5768,CA_BAB_CA_HOST=SAEL-PROD-DB1,CA_BAB_SOURCE_MACHINE_REGIST
2010-03-24 23:35:51.724 [[ 5732]:( 5768)] ORABL[ERR]6> ERED_NAME=SAEL-PROD-DB1,CA_BAB_ABORT_EVENT=52cea716-b5b2-42be-b724-bb138b02a0a3,CA_BAB_MACHINE_USER_NAME=Sael-Prod-db1\Administrator,CA_BAB_MACHINE_PWD=UlQAAQAAAAoAAAACAAAAIAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAN9xErlkn0K+vGbjLY+J28Kdhorwtf42z8
2010-03-24 23:35:51.724 [[ 5732]:( 5768)] ORABL[ERR]7> apyrQQfggQ,CA_BAB_DB_USER_NAME=Sael-Prod-db1\Administrator,CA_BAB_DB_PWD=UlQAAQAAAAoAAAACAAAAIAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAN9xErlkn0K+vGbjLY+J28Kdhorwtf42z8apyrQQfggQ,CA_BAB_SECURITY_ID=UlQAAQAAAAoAAAACAAAAYAAAAAAAAAAAAAAAAAAAAAAAAAAAA
2010-03-24 23:35:51.724 [[ 5732]:( 5768)] ORABL[ERR]8> AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABScQ8fOGDdKxyXC6Sunzi0Lo/nQCEweMnwNwIASaww67vGIuO4D0PbCFJKl+fDx3DKvAR8SNyZqKSpdrujdCdtsDPXSF/pyTAQhA1LiFqaXG2Z19aaUw0kD9/ccCc/L9Q=) ';
2010-03-24 23:35:51.724 [[ 5732]:( 5768)] ORABL[ERR]9> ALLOCATE CHANNEL 'cccora02' DEVICE TYPE sbt FORMAT '_8c7ca61f-6c56-4fc3-8063-cc412a5363ed_%u_%p_%c_' PARMS=' ENV=(CA_BAB_MASTER_JOBID=27,CA_BAB_DUMMY_JOBID=2,CA_BAB_PROCESS_ID=2027284384,CA_BAB_THREAD_ID=5768,CA_BAB_CA_HOST=SAEL-PROD-DB1,CA_BAB_SOURCE_MACHINE_REGIST
2010-03-24 23:35:51.724 [[ 5732]:( 5768)] ORABL[ERR]10> ERED_NAME=SAEL-PROD-DB1,CA_BAB_ABORT_EVENT=52cea716-b5b2-42be-b724-bb138b02a0a3,CA_BAB_MACHINE_USER_NAME=Sael-Prod-db1\Administrator,CA_BAB_MACHINE_PWD=UlQAAQAAAAoAAAACAAAAIAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAN9xErlkn0K+vGbjLY+J28Kdhorwtf42z8
2010-03-24 23:35:51.724 [[ 5732]:( 5768)] ORABL[ERR]11> apyrQQfggQ,CA_BAB_DB_USER_NAME=Sael-Prod-db1\Administrator,CA_BAB_DB_PWD=UlQAAQAAAAoAAAACAAAAIAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAN9xErlkn0K+vGbjLY+J28Kdhorwtf42z8apyrQQfggQ,CA_BAB_SECURITY_ID=UlQAAQAAAAoAAAACAAAAYAAAAAAAAAAAAAAAAAAAAAAAAAAAA
2010-03-24 23:35:51.724 [[ 5732]:( 5768)] ORABL[ERR]12> AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABScQ8fOGDdKxyXC6Sunzi0Lo/nQCEweMnwNwIASaww67vGIuO4D0PbCFJKl+fDx3DKvAR8SNyZqKSpdrujdCdtsDPXSF/pyTAQhA1LiFqaXG2Z19aaUw0kD9/ccCc/L9Q=) ';
2010-03-24 23:35:51.724 [[ 5732]:( 5768)] ORABL[ERR]13> BACKUP AS COMPRESSED BACKUPSET INCREMENTAL LEVEL 1 ( DATABASE ) ;
2010-03-24 23:35:51.724 [[ 5732]:( 5768)] ORABL[ERR]14> BACKUP AS COMPRESSED BACKUPSET INCREMENTAL LEVEL 1 ( CURRENT CONTROLFILE ) ;
2010-03-24 23:35:51.724 [[ 5732]:( 5768)] ORABL[ERR]15> SQL 'ALTER SYSTEM ARCHIVE LOG CURRENT';
2010-03-24 23:35:51.724 [[ 5732]:( 5768)] ORABL[ERR]16> BACKUP AS COMPRESSED BACKUPSET INCREMENTAL LEVEL 1 ( ARCHIVELOG ALL DELETE ALL INPUT ) ;
2010-03-24 23:35:51.724 [[ 5732]:( 5768)] ORABL[ERR]17>
2010-03-24 23:35:51.724 [[ 5732]:( 5768)] ORABL[ERR]18> }
2010-03-24 23:35:51.724 [[ 5732]:( 5768)] ORABL[ERR]19> **end-of-file**
2010-03-24 23:35:51.724 [[ 5732]:( 5768)] ORABL[ERR]20>
2010-03-24 23:35:51.724 [[ 5732]:( 5768)] ORABL[ERR]connected to target database: CCDBPROD (DBID=1520750004)
2010-03-24 23:35:51.724 [[ 5732]:( 5768)] ORABL[ERR]connected to recovery catalog database
2010-03-24 23:35:51.724 [[ 5732]:( 5768)] ORABL[ERR]allocated channel: cccora01
2010-03-24 23:35:51.724 [[ 5732]:( 5768)] ORABL[ERR]channel cccora01: sid=1376 instance=CCDBPROD1 devtype=SBT_TAPE
2010-03-24 23:35:51.724 [[ 5732]:( 5768)] ORABL[ERR]channel cccora01: CA ARCserve Backup Oracle Agent v12.5
2010-03-24 23:35:51.724 [[ 5732]:( 5768)] ORABL[ERR]allocated channel: cccora02
2010-03-24 23:35:51.724 [[ 5732]:( 5768)] ORABL[ERR]channel cccora02: sid=1398 instance=CCDBPROD1 devtype=SBT_TAPE
2010-03-24 23:35:51.724 [[ 5732]:( 5768)] ORABL[ERR]channel cccora02: CA ARCserve Backup Oracle Agent v12.5
2010-03-24 23:35:51.724 [[ 5732]:( 5768)] ORABL[ERR]Starting backup at 24-MAR-10
2010-03-24 23:35:51.740 [[ 5732]:( 5768)] ORABL[ERR]channel cccora01: starting compressed incremental level 1 datafile backupset
2010-03-24 23:35:51.740 [[ 5732]:( 5768)] ORABL[ERR]channel cccora01: specifying datafile(s) in backupset
2010-03-24 23:35:51.740 [[ 5732]:( 5768)] ORABL[ERR]input datafile fno=00008 name=+DATA/CCDBPROD/datafile/lstar_data.dbf
2010-03-24 23:35:51.740 [[ 5732]:( 5768)] ORABL[ERR]input datafile fno=00003 name=+DATA/CCDBPROD/datafile/sysaux.262.692467787
2010-03-24 23:35:51.740 [[ 5732]:( 5768)] ORABL[ERR]input datafile fno=00013 name=+DATA/CCDBPROD/datafile/sysaux.extended
2010-03-24 23:35:51.740 [[ 5732]:( 5768)] ORABL[ERR]input datafile fno=00021 name=+DATA/CCDBPROD/datafile/sysaux.286.700135361
2010-03-24 23:35:51.740 [[ 5732]:( 5768)] ORABL[ERR]input datafile fno=00022 name=+DATA/CCDBPROD/datafile/sysaux.287.700656779
2010-03-24 23:35:51.740 [[ 5732]:( 5768)] ORABL[ERR]input datafile fno=00023 name=+DATA/CCDBPROD/datafile/sysaux.288.700826653
2010-03-24 23:35:51.740 [[ 5732]:( 5768)] ORABL[ERR]input datafile fno=00025 name=+DATA/CCDBPROD/datafile/sysaux.292.703164965
2010-03-24 23:35:51.740 [[ 5732]:( 5768)] ORABL[ERR]input datafile fno=00001 name=+DATA/CCDBPROD/datafile/system.260.692467775
2010-03-24 23:35:51.740 [[ 5732]:( 5768)] ORABL[ERR]input datafile fno=00011 name=+DATA/CCDBPROD/datafile/system.extend
2010-03-24 23:35:51.740 [[ 5732]:( 5768)] ORABL[ERR]input datafile fno=00024 name=+DATA/CCDBPROD/datafile/system.289.700826697
2010-03-24 23:35:51.740 [[ 5732]:( 5768)] ORABL[ERR]input datafile fno=00005 name=+DATA/CCDBPROD/datafile/users.265.692467799
2010-03-24 23:35:51.740 [[ 5732]:( 5768)] ORABL[ERR]channel cccora01: starting piece 1 at 24-MAR-10
2010-03-24 23:35:51.740 [[ 5732]:( 5768)] ORABL[ERR]channel cccora02: starting compressed incremental level 1 datafile backupset
2010-03-24 23:35:51.740 [[ 5732]:( 5768)] ORABL[ERR]channel cccora02: specifying datafile(s) in backupset
2010-03-24 23:35:51.740 [[ 5732]:( 5768)] ORABL[ERR]input datafile fno=00009 name=+DATA/CCDBPROD/datafile/lstar_index.dbf
2010-03-24 23:35:51.740 [[ 5732]:( 5768)] ORABL[ERR]input datafile fno=00010 name=+DATA/CCDBPROD/datafile/lstar_lob.dbf
2010-03-24 23:35:51.740 [[ 5732]:( 5768)] ORABL[ERR]input datafile fno=00004 name=+DATA/CCDBPROD/datafile/undotbs2.264.692467797
2010-03-24 23:35:51.740 [[ 5732]:( 5768)] ORABL[ERR]input datafile fno=00002 name=+DATA/CCDBPROD/datafile/undotbs1.261.692467785
2010-03-24 23:35:51.740 [[ 5732]:( 5768)] ORABL[ERR]input datafile fno=00017 name=+DATA/CCDBPROD/datafile/undotbs2b
2010-03-24 23:35:51.740 [[ 5732]:( 5768)] ORABL[ERR]input datafile fno=00014 name=+DATA/CCDBPROD/datafile/undotbs2
2010-03-24 23:35:51.740 [[ 5732]:( 5768)] ORABL[ERR]input datafile fno=00015 name=+DATA/CCDBPROD/datafile/undotbs1b.extend
2010-03-24 23:35:51.740 [[ 5732]:( 5768)] ORABL[ERR]input datafile fno=00012 name=+DATA/CCDBPROD/datafile/undotbs1.extend
2010-03-24 23:35:51.740 [[ 5732]:( 5768)] ORABL[ERR]input datafile fno=00016 name=+DATA/CCDBPROD/datafile/undotbs1c.extend
2010-03-24 23:35:51.740 [[ 5732]:( 5768)] ORABL[ERR]input datafile fno=00018 name=+DATA/CCDBPROD/datafile/undotbs1d.extend
2010-03-24 23:35:51.740 [[ 5732]:( 5768)] ORABL[ERR]input datafile fno=00019 name=+DATA/CCDBPROD/datafile/undotbs1e.extend
2010-03-24 23:35:51.740 [[ 5732]:( 5768)] ORABL[ERR]input datafile fno=00020 name=+DATA/CCDBPROD/datafile/undotbs1f.extend
2010-03-24 23:35:51.740 [[ 5732]:( 5768)] ORABL[ERR]channel cccora02: starting piece 1 at 24-MAR-10
2010-03-24 23:35:51.740 [[ 5732]:( 5768)] ORABL[ERR]channel cccora02: finished piece 1 at 24-MAR-10
2010-03-24 23:35:51.740 [[ 5732]:( 5768)] ORABL[ERR]piece handle=_8c7ca61f-6c56-4fc3-8063-cc412a5363ed_rcl9de8b_1_1_ tag=TAG20100324T220043 comment=API Version 2.0,MMS Version 12.5.0.0
2010-03-24 23:35:51.740 [[ 5732]:( 5768)] ORABL[ERR]channel cccora02: backup set complete, elapsed time: 00:22:45
2010-03-24 23:35:51.740 [[ 5732]:( 5768)] ORABL[ERR]channel cccora02: starting compressed incremental level 1 datafile backupset
2010-03-24 23:35:51.740 [[ 5732]:( 5768)] ORABL[ERR]channel cccora02: specifying datafile(s) in backupset
2010-03-24 23:35:51.740 [[ 5732]:( 5768)] ORABL[ERR]input datafile fno=00007 name=+DATA/CCDBPROD/datafile/acea_rep.dbf
2010-03-24 23:35:51.740 [[ 5732]:( 5768)] ORABL[ERR]channel cccora02: starting piece 1 at 24-MAR-10
2010-03-24 23:35:51.740 [[ 5732]:( 5768)] ORABL[ERR]channel cccora02: finished piece 1 at 24-MAR-10
2010-03-24 23:35:51.740 [[ 5732]:( 5768)] ORABL[ERR]piece handle=_8c7ca61f-6c56-4fc3-8063-cc412a5363ed_rdl9dfj1_1_1_ tag=TAG20100324T220043 comment=API Version 2.0,MMS Version 12.5.0.0
2010-03-24 23:35:51.740 [[ 5732]:( 5768)] ORABL[ERR]channel cccora02: backup set complete, elapsed time: 00:00:55
2010-03-24 23:35:51.740 [[ 5732]:( 5768)] ORABL[ERR]channel cccora02: starting compressed incremental level 1 datafile backupset
2010-03-24 23:35:51.740 [[ 5732]:( 5768)] ORABL[ERR]channel cccora02: specifying datafile(s) in backupset
2010-03-24 23:35:51.740 [[ 5732]:( 5768)] ORABL[ERR]input datafile fno=00006 name=+DATA/CCDBPROD/datafile/acea_index_rep.dbf
2010-03-24 23:35:51.740 [[ 5732]:( 5768)] ORABL[ERR]channel cccora02: starting piece 1 at 24-MAR-10
2010-03-24 23:35:51.740 [[ 5732]:( 5768)] ORABL[ERR]channel cccora02: finished piece 1 at 24-MAR-10
2010-03-24 23:35:51.740 [[ 5732]:( 5768)] ORABL[ERR]piece handle=_8c7ca61f-6c56-4fc3-8063-cc412a5363ed_rel9dfko_1_1_ tag=TAG20100324T220043 comment=API Version 2.0,MMS Version 12.5.0.0
2010-03-24 23:35:51.740 [[ 5732]:( 5768)] ORABL[ERR]channel cccora02: backup set complete, elapsed time: 00:00:35
2010-03-24 23:35:51.740 [[ 5732]:( 5768)] ORABL[ERR]channel cccora02: starting compressed incremental level 1 datafile backupset
2010-03-24 23:35:51.740 [[ 5732]:( 5768)] ORABL[ERR]channel cccora02: specifying datafile(s) in backupset
2010-03-24 23:35:51.740 [[ 5732]:( 5768)] ORABL[ERR]including current control file in backupset
2010-03-24 23:35:51.740 [[ 5732]:( 5768)] ORABL[ERR]channel cccora02: starting piece 1 at 24-MAR-10
2010-03-24 23:35:51.740 [[ 5732]:( 5768)] ORABL[ERR]channel cccora02: finished piece 1 at 24-MAR-10
2010-03-24 23:35:51.740 [[ 5732]:( 5768)] ORABL[ERR]piece handle=_8c7ca61f-6c56-4fc3-8063-cc412a5363ed_rfl9dflr_1_1_ tag=TAG20100324T220043 comment=API Version 2.0,MMS Version 12.5.0.0
2010-03-24 23:35:51.740 [[ 5732]:( 5768)] ORABL[ERR]channel cccora02: backup set complete, elapsed time: 00:00:08
2010-03-24 23:35:51.740 [[ 5732]:( 5768)] ORABL[ERR]channel cccora02: starting compressed incremental level 1 datafile backupset
2010-03-24 23:35:51.756 [[ 5732]:( 5768)] ORABL[ERR]channel cccora02: specifying datafile(s) in backupset
2010-03-24 23:35:51.756 [[ 5732]:( 5768)] ORABL[ERR]including current SPFILE in backupset
2010-03-24 23:35:51.756 [[ 5732]:( 5768)] ORABL[ERR]channel cccora02: starting piece 1 at 24-MAR-10
2010-03-24 23:35:51.756 [[ 5732]:( 5768)] ORABL[ERR]channel cccora02: finished piece 1 at 24-MAR-10
2010-03-24 23:35:51.756 [[ 5732]:( 5768)] ORABL[ERR]piece handle=_8c7ca61f-6c56-4fc3-8063-cc412a5363ed_rgl9dfm3_1_1_ tag=TAG20100324T220043 comment=API Version 2.0,MMS Version 12.5.0.0
2010-03-24 23:35:51.756 [[ 5732]:( 5768)] ORABL[ERR]channel cccora02: backup set complete, elapsed time: 00:00:02
2010-03-24 23:35:51.756 [[ 5732]:( 5768)] ORABL[ERR]RMAN-03009: failure of backup command on cccora01 channel at 03/24/2010 22:54:34
2010-03-24 23:35:51.756 [[ 5732]:( 5768)] ORABL[ERR]ORA-19502: write error on file "_8edd9910-cfe5-45e3-a7a2-fc1939dd152f_rbl9de8b_1_1_", blockno 39425 (blocksize=512)
2010-03-24 23:35:51.756 [[ 5732]:( 5768)] ORABL[ERR]ORA-27030: skgfwrt: sbtwrite2 returned error
2010-03-24 23:35:51.756 [[ 5732]:( 5768)] ORABL[ERR]channel cccora01 disabled, job failed on it will be run on another channel
2010-03-24 23:35:51.756 [[ 5732]:( 5768)] ORABL[ERR]channel cccora02: starting compressed incremental level 1 datafile backupset
2010-03-24 23:35:51.756 [[ 5732]:( 5768)] ORABL[ERR]channel cccora02: specifying datafile(s) in backupset
2010-03-24 23:35:51.756 [[ 5732]:( 5768)] ORABL[ERR]input datafile fno=00008 name=+DATA/CCDBPROD/datafile/lstar_data.dbf
2010-03-24 23:35:51.756 [[ 5732]:( 5768)] ORABL[ERR]input datafile fno=00003 name=+DATA/CCDBPROD/datafile/sysaux.262.692467787
2010-03-24 23:35:51.756 [[ 5732]:( 5768)] ORABL[ERR]input datafile fno=00013 name=+DATA/CCDBPROD/datafile/sysaux.extended
2010-03-24 23:35:51.756 [[ 5732]:( 5768)] ORABL[ERR]input datafile fno=00021 name=+DATA/CCDBPROD/datafile/sysaux.286.700135361
2010-03-24 23:35:51.756 [[ 5732]:( 5768)] ORABL[ERR]input datafile fno=00022 name=+DATA/CCDBPROD/datafile/sysaux.287.700656779
2010-03-24 23:35:51.756 [[ 5732]:( 5768)] ORABL[ERR]input datafile fno=00023 name=+DATA/CCDBPROD/datafile/sysaux.288.700826653
2010-03-24 23:35:51.756 [[ 5732]:( 5768)] ORABL[ERR]input datafile fno=00025 name=+DATA/CCDBPROD/datafile/sysaux.292.703164965
2010-03-24 23:35:51.756 [[ 5732]:( 5768)] ORABL[ERR]input datafile fno=00001 name=+DATA/CCDBPROD/datafile/system.260.692467775
2010-03-24 23:35:51.756 [[ 5732]:( 5768)] ORABL[ERR]input datafile fno=00011 name=+DATA/CCDBPROD/datafile/system.extend
2010-03-24 23:35:51.756 [[ 5732]:( 5768)] ORABL[ERR]input datafile fno=00024 name=+DATA/CCDBPROD/datafile/system.289.700826697
2010-03-24 23:35:51.756 [[ 5732]:( 5768)] ORABL[ERR]input datafile fno=00005 name=+DATA/CCDBPROD/datafile/users.265.692467799
2010-03-24 23:35:51.756 [[ 5732]:( 5768)] ORABL[ERR]channel cccora02: starting piece 1 at 24-MAR-10
2010-03-24 23:35:51.756 [[ 5732]:( 5768)] ORABL[ERR]released channel: cccora01
2010-03-24 23:35:51.756 [[ 5732]:( 5768)] ORABL[ERR]released channel: cccora02
2010-03-24 23:35:51.756 [[ 5732]:( 5768)] ORABL[ERR]RMAN-00571: ===========================================================
2010-03-24 23:35:51.756 [[ 5732]:( 5768)] ORABL[ERR]RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============
2010-03-24 23:35:51.756 [[ 5732]:( 5768)] ORABL[ERR]RMAN-00571: ===========================================================
2010-03-24 23:35:51.756 [[ 5732]:( 5768)] ORABL[ERR]RMAN-03009: failure of backup command on cccora02 channel at 03/24/2010 23:35:50
2010-03-24 23:35:51.756 [[ 5732]:( 5768)] ORABL[ERR]ORA-19502: write error on file "_8c7ca61f-6c56-4fc3-8063-cc412a5363ed_rhl9dhda_1_1_", blockno 39937 (blocksize=512)
2010-03-24 23:35:51.756 [[ 5732]:( 5768)] ORABL[ERR]ORA-27030: skgfwrt: sbtwrite2 returned error
2010-03-24 23:35:51.756 [[ 5732]:( 5768)] ORABL[ERR]Recovery Manager complete.
2010-03-24 23:35:51.756 [[ 5732]:( 5768)] ORACMD[INFO]After run command, Current user is Administrator
2010-03-24 23:35:51.756 [[ 5732]:( 5768)] ORAADP[INFO]submitRMANBackup
2010-03-24 23:35:51.756 [[ 5732]:( 5768)] ORAADP[ERR]submitRMANBackup failed 65537
we read around the problem and found in the del metalink ID 412950.1 that it is often caused by the media management errors.
The strange thing is that after the ORA-27030 message, we should have the detailed media trace, and instead we don't get anything.
We also verified, and the space on the disk wasn't filled up.
Any help would be appreciated.