Hi,
We're getting a restore problem seems like a random occurrence of RMAN 6956 create datafile failed.
The production database files are on ASM file system and we're restoring the database to non-ASM, regular file system.
THe database version is 11.2.0.4 running on windows2012 while we are trying to restore the database on a windows2016 server.
THe production database and the new target host has the same patch level.
The file names will be renamed to regular OS files. Sample restore script below:
run {
allocate channel d1 type disk;
allocate channel d2 type disk;
allocate channel d3 type disk;
allocate channel d4 type disk;
set newname for datafile 12 to 'F:\ORADATA\ICARITAS\DATAFILE\lind02.429.861229823';
set newname for datafile 7 to 'F:\ORADATA\ICARITAS\DATAFILE\lind03.299.861253411';
set newname for datafile 8 to 'F:\ORADATA\ICARITAS\DATAFILE\lind04.628.861240339';
set newname for datafile 24 to 'F:\ORADATA\ICARITAS\DATAFILE\lind07.358.861247879';
set newname for datafile 11 to 'F:\ORADATA\ICARITAS\DATAFILE\lind09.706.861229631';
set newname for datafile 208 to 'F:\ORADATA\ICARITAS\DATAFILE\lind10.503.861240339';
set newname for datafile 213 to 'F:\ORADATA\ICARITAS\DATAFILE\lind12.498.861224827';
set newname for datafile 176 to 'F:\ORADATA\ICARITAS\DATAFILE\lind14.615.861234505';
set newname for datafile 16 to 'F:\ORADATA\ICARITAS\DATAFILE\lind16.373.861232545';
set newname for datafile 49 to 'F:\ORADATA\ICARITAS\DATAFILE\lind18.611.861234799';
set newname for datafile 18 to 'F:\ORADATA\ICARITAS\DATAFILE\lind19.341.861235055';
set newname for datafile 199 to 'F:\ORADATA\ICARITAS\DATAFILE\lind21.697.861248109';
set newname for datafile 264 to 'F:\ORADATA\ICARITAS\DATAFILE\lind22.682.861222015';
set newname for datafile 211 to 'F:\ORADATA\ICARITAS\DATAFILE\lind25.561.861227367';
set newname for datafile 258 to 'F:\ORADATA\ICARITAS\DATAFILE\lind25.594.861242949';
set newname for datafile 226 to 'F:\ORADATA\ICARITAS\DATAFILE\lind28.310.861219599';
set newname for datafile 269 to 'F:\ORADATA\ICARITAS\DATAFILE\lind28.644.861248111';
set newname for datafile 316 to 'F:\ORADATA\ICARITAS\DATAFILE\lind29.425.861244285';
set newname for datafile 332 to 'F:\ORADATA\ICARITAS\DATAFILE\lind36.302.861248161';
......continued......
set newname for datafile 481 to 'F:\ORADATA\ICARITAS\DATAFILE\ltab74.332.982345589';
set newname for datafile 581 to 'F:\ORADATA\ICARITAS\DATAFILE\ltab75.333.982345889';
set newname for datafile 610 to 'F:\ORADATA\ICARITAS\DATAFILE\ltab81.334.982414055';
set newname for datafile 611 to 'F:\ORADATA\ICARITAS\DATAFILE\ltab82.335.982414405';
set newname for tempfile 1 to 'F:\ORADATA\ICARITAS\TEMPFILE\temp01.651.861322391';
set newname for tempfile 2 to 'F:\ORADATA\ICARITAS\TEMPFILE\temp02.408.861322391';
set newname for tempfile 3 to 'F:\ORADATA\ICARITAS\TEMPFILE\temp03.266.922982157';
restore database check readonly;
SWITCH DATAFILE ALL;
SWITCH TEMPFILE ALL;
release channel d1;
release channel d2;
release channel d3;
release channel d4;
}
Actual error messages on the restore:
RMAN-00571: ===========================================================
RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============
RMAN-00571: ===========================================================
RMAN-03002: failure of restore command at 03/23/2019 15:27:25
ORA-01119: error in creating database file '+asmd3dskgrp1'
ORA-17502: ksfdcre:4 Failed to create file +asmd3dskgrp1
ORA-15001: diskgroup "ASMD3DSKGRP1" does not exist or is not mounted
ORA-15077: could not locate ASM instance serving a required diskgroup
ORA-29701: unable to connect to Cluster Synchronization Service
RMAN-06956: create datafile failed; retry after removing F:\ORADATA\ICARITAS\DATAFILE\ltab18.335.861219283 from OS
RMAN> list backup of datafile 17;
List of Backup Sets
===================
BS Key Type LV Size Device Type Elapsed Time Completion Time
------- ---- -- ---------- ----------- ------------ ---------------
59649 Incr 1 218.51M DISK 00:10:40 05-MAR-19
BP Key: 259038 Status: EXPIRED Compressed: NO Tag: TAG20190305T230012
Piece Name: I:\ICARITAS_RMAN\ICARITAS_INCREMBKPC_7HTRN8D711
List of Datafiles in backup set 59649
File LV Type Ckp SCN Ckp Time Name
---- -- ---- ---------- --------- ----
17 1 Incr 2956861871 05-MAR-19 +ASMD3DSKGRP1/icaritas/datafile/ltab18.335.861219283
BS Key Type LV Size Device Type Elapsed Time Completion Time
------- ---- -- ---------- ----------- ------------ ---------------
59672 Incr 1 240.08M DISK 00:09:54 06-MAR-19
BP Key: 259061 Status: EXPIRED Compressed: NO Tag: TAG20190306T230012
Piece Name: I:\ICARITAS_RMAN\ICARITAS_INCREMBKPC_88TRPSS111
List of Datafiles in backup set 59672
File LV Type Ckp SCN Ckp Time Name
---- -- ---- ---------- --------- ----
17 1 Incr 2957543716 06-MAR-19 +ASMD3DSKGRP1/icaritas/datafile/ltab18.335.861219283
BS Key Type LV Size Device Type Elapsed Time Completion Time
------- ---- -- ---------- ----------- ------------ ---------------
59696 Incr 1 242.48M DISK 00:13:15 07-MAR-19
BP Key: 259085 Status: EXPIRED Compressed: NO Tag: TAG20190307T091301
Piece Name: I:\ICARITAS_RMAN\ICARITAS_INCREMBKPC_98TRR15P11
List of Datafiles in backup set 59696
File LV Type Ckp SCN Ckp Time Name
---- -- ---- ---------- --------- ----
17 1 Incr 2957682299 07-MAR-19 +ASMD3DSKGRP1/icaritas/datafile/ltab18.335.861219283
BS Key Type LV Size Device Type Elapsed Time Completion Time
------- ---- -- ---------- ----------- ------------ ---------------
59721 Incr 1 232.94M DISK 00:11:47 08-MAR-19
BP Key: 259110 Status: EXPIRED Compressed: NO Tag: TAG20190307T230012
Piece Name: I:\ICARITAS_RMAN\ICARITAS_INCREMBKPC_A5TRSHQ311
List of Datafiles in backup set 59721
File LV Type Ckp SCN Ckp Time Name
---- -- ---- ---------- --------- ----
17 1 Incr 2958171431 07-MAR-19 +ASMD3DSKGRP1/icaritas/datafile/ltab18.335.861219283
BS Key Type LV Size Device Type Elapsed Time Completion Time
------- ---- -- ---------- ----------- ------------ ---------------
59770 Incr 0 74.62G DISK 01:09:51 10-MAR-19
List of Datafiles in backup set 59770
File LV Type Ckp SCN Ckp Time Name
---- -- ---- ---------- --------- ----
17 0 Incr 2959289922 09-MAR-19 +ASMD3DSKGRP1/icaritas/datafile/ltab18.335.861219283
Backup Set Copy #1 of backup set 59770
Device Type Elapsed Time Completion Time Compressed Tag
----------- ------------ --------------- ---------- ---
DISK 01:09:51 10-MAR-19 NO TAG20190309T192034
List of Backup Pieces for backup set 59770 Copy #1
BP Key Pc# Status Piece Name
------- --- ----------- ----------
259267 1 EXPIRED J:\ICARITAS_RMAN\ICARITAS_BKPC_B4TS1QJ711
259268 2 EXPIRED J:\ICARITAS_RMAN\ICARITAS_BKPC_B4TS1QJ721
259269 3 EXPIRED J:\ICARITAS_RMAN\ICARITAS_BKPC_B4TS1QJ731
259270 4 EXPIRED J:\ICARITAS_RMAN\ICARITAS_BKPC_B4TS1QJ741
259271 5 EXPIRED J:\ICARITAS_RMAN\ICARITAS_BKPC_B4TS1QJ751
259272 6 EXPIRED J:\ICARITAS_RMAN\ICARITAS_BKPC_B4TS1QJ761
259273 7 EXPIRED J:\ICARITAS_RMAN\ICARITAS_BKPC_B4TS1QJ771
259274 8 EXPIRED J:\ICARITAS_RMAN\ICARITAS_BKPC_B4TS1QJ781
259275 9 EXPIRED J:\ICARITAS_RMAN\ICARITAS_BKPC_B4TS1QJ791
259276 10 EXPIRED J:\ICARITAS_RMAN\ICARITAS_BKPC_B4TS1QJ7101
BS Key Type LV Size Device Type Elapsed Time Completion Time
------- ---- -- ---------- ----------- ------------ ---------------
59789 Incr 1 38.09M DISK 00:15:46 12-MAR-19
BP Key: 259348 Status: EXPIRED Compressed: NO Tag: TAG20190312T082305
Piece Name: I:\ICARITAS_RMAN\ICARITAS_INCREMBKPC_BSTS84P211
List of Datafiles in backup set 59789
File LV Type Ckp SCN Ckp Time Name
---- -- ---- ---------- --------- ----
17 1 Incr 2960322008 12-MAR-19 +ASMD3DSKGRP1/icaritas/datafile/ltab18.335.861219283
BS Key Type LV Size Device Type Elapsed Time Completion Time
------- ---- -- ---------- ----------- ------------ ---------------
59807 Incr 1 121.41M DISK 00:11:05 12-MAR-19
BP Key: 259366 Status: EXPIRED Compressed: NO Tag: TAG20190312T230012
Piece Name: I:\ICARITAS_RMAN\ICARITAS_INCREMBKPC_CDTS9LDU11
List of Datafiles in backup set 59807
File LV Type Ckp SCN Ckp Time Name
---- -- ---- ---------- --------- ----
17 1 Incr 2961045681 12-MAR-19 +ASMD3DSKGRP1/icaritas/datafile/ltab18.335.861219283
BS Key Type LV Size Device Type Elapsed Time Completion Time
------- ---- -- ---------- ----------- ------------ ---------------
59836 Incr 1 610.38M DISK 00:12:00 14-MAR-19
BP Key: 259395 Status: EXPIRED Compressed: NO Tag: TAG20190313T230009
Piece Name: I:\ICARITAS_RMAN\ICARITAS_INCREMBKPC_DATSCC5811
List of Datafiles in backup set 59836
File LV Type Ckp SCN Ckp Time Name
---- -- ---- ---------- --------- ----
17 1 Incr 2961792284 13-MAR-19 +ASMD3DSKGRP1/icaritas/datafile/ltab18.335.861219283
BS Key Type LV Size Device Type Elapsed Time Completion Time
------- ---- -- ---------- ----------- ------------ ---------------
59859 Incr 1 554.18M DISK 00:15:03 15-MAR-19
BP Key: 259418 Status: EXPIRED Compressed: NO Tag: TAG20190314T230014
Piece Name: I:\ICARITAS_RMAN\ICARITAS_INCREMBKPC_E1TSF0CP11
List of Datafiles in backup set 59859
File LV Type Ckp SCN Ckp Time Name
---- -- ---- ---------- --------- ----
17 1 Incr 2962392508 14-MAR-19 +ASMD3DSKGRP1/icaritas/datafile/ltab18.335.861219283
BS Key Type LV Size Device Type Elapsed Time Completion Time
------- ---- -- ---------- ----------- ------------ ---------------
59873 Incr 0 75.47G DISK 01:04:18 16-MAR-19
List of Datafiles in backup set 59873
File LV Type Ckp SCN Ckp Time Name
---- -- ---- ---------- --------- ----
17 0 Incr 2963361799 16-MAR-19 +ASMD3DSKGRP1/icaritas/datafile/ltab18.335.861219283
Backup Set Copy #1 of backup set 59873
Device Type Elapsed Time Completion Time Compressed Tag
----------- ------------ --------------- ---------- ---
DISK 01:04:18 16-MAR-19 NO TAG20190316T200110
List of Backup Pieces for backup set 59873 Copy #1
BP Key Pc# Status Piece Name
------- --- ----------- ----------
259432 1 AVAILABLE I:\ICARITAS_RMAN\ICARITAS_BKPC_EJTSJROK11
259433 2 AVAILABLE I:\ICARITAS_RMAN\ICARITAS_BKPC_EJTSJROK21
259434 3 AVAILABLE I:\ICARITAS_RMAN\ICARITAS_BKPC_EJTSJROK31
259435 4 AVAILABLE I:\ICARITAS_RMAN\ICARITAS_BKPC_EJTSJROK41
259436 5 AVAILABLE I:\ICARITAS_RMAN\ICARITAS_BKPC_EJTSJROK51
259437 6 AVAILABLE I:\ICARITAS_RMAN\ICARITAS_BKPC_EJTSJROK61
259438 7 AVAILABLE I:\ICARITAS_RMAN\ICARITAS_BKPC_EJTSJROK71
259439 8 AVAILABLE I:\ICARITAS_RMAN\ICARITAS_BKPC_EJTSJROK81
259440 9 AVAILABLE I:\ICARITAS_RMAN\ICARITAS_BKPC_EJTSJROK91
259441 10 AVAILABLE I:\ICARITAS_RMAN\ICARITAS_BKPC_EJTSJROK101
RMAN> exit
Hope you can help us troubleshoot,
Regards