In-Frequently, Oracle 10g 64-bits DB's backups are getting failed with error message"ORA-27192: skgfcls: sbtclose2 returned error - failed to close file" followed by "ORA-19511: Error received from media manager layer" The cause being:"RPC receive operation failed. A network connection could not be established with the host". On all the 4 channels which is configured in our RMAN scripts, Which looks somthing like this:
# vi rmanweeklyfull.cmd
"rmanweeklyfull.cmd" 26 lines, 928 characters
connect target sys/<*>@DWPROD;
CONFIGURE DEVICE TYPE sbt PARALLELISM 4;
CONFIGURE RETENTION POLICY TO RECOVERY WINDOW OF 30 DAYS;
run {
allocate channel t1 type 'SBT_TAPE';
allocate channel t2 type 'SBT_TAPE';
allocate channel t3 type 'SBT_TAPE';
allocate channel t4 type 'SBT_TAPE';
send 'NSR_ENV=(NSR_SERVER=<backup_server_name> ,NSR_CLIENT=<server_name>,NSR_GROUP=Weekly Full DWBI DB, NSR_VOLUME_POOL=DWBI)';
backup full filesperset 4 format 'DB_FULL_%d_%U_%T' (database);
sql 'alter system switch logfile';
sql 'alter system archive log current';
crosscheck archivelog all;
backup format 'DB_ARCH_%s_%p_%T' archivelog all;
backup format 'cntrl_s%s_p%p_%T' current controlfile;
backup format 'spfile_s%s_p%p_%T' spfile;
DELETE noprompt ARCHIVELOG ALL BACKED UP 2 TIMES TO DEVICE TYPE sbt;
release channel t4;
release channel t3;
release channel t2;
release channel t1;
}
We are using Legheto/EMC Networker v7.6 SP3, As it is, from the last line in error message it points out to be a RPC network connection related error. But , I will like to get some help in identifying the root cause for this error-Why exactly it surfaces in-frequently? as I have already got some references from closed tread link:RMAN-03009 ORA-27192 ORA-19511 VxBSAEndTxn: Failed with error:
Our 'ulimit' values for this server are:
# ulimit -a
time(seconds) unlimited
file(blocks) unlimited
data(kbytes) unlimited
stack(kbytes) 4194304
memory(kbytes) unlimited
coredump(blocks) 0
nofiles(descriptors) 65536
We are using a FC 4gBps SAN network directly connected to an VTL [EMC Disk Library-4206 model, configured in EMC Clariion 80-3f] tapes from client DB server, setup for the flow of backup data to the Storage.
I will welcome any suggestions/pointers towards technically analysing this error!
I will like to know the purpose of "backup full filesperset 4" in the RMAN script , what's 'fileperset' value used in for? {Can we change it to check for any difference, or is it recommended to be left in vendor specified state i.e 4 in this case?}
Lastly, in OS server end what all parameters to be checked for regarding RPC related concerns[please also mention the exact commands].
I will like to Thank anyone who takes the trouble of going though this long query and respond to it in advance, I want you to know That I really appreciate your gesture.