General RAC database question
765364Sep 14 2010 — edited Sep 14 2010Hi,
I am a Junior DBA and I am new to Oracle RAC. I have a quick question about an Oracle RAC database that I'm maintaining. Over the weekend a network card was pulled and one of the database nodes went down. When the network card was plugged back in and network connection was returned, the database node stayed down. The following error was found continuously in the crsd.log file:
2010-09-11 10:05:15.286: [ OCRRAW][258540112]proprioini: disk 0 (/vote1/storage/ocr) doesn't have enough votes (1,2)
2010-09-11 10:05:15.286: [ OCRRAW][258540112]proprioo: Not enought quorum to open the disks (26)
2010-09-11 10:05:15.286: [ OCRRAW][258540112]proprinit: Could not open raw device
2010-09-11 10:05:15.286: [ default][258540112]a_init:7!: Backend init unsuccessful : [26]
2010-09-11 10:05:16.290: [ OCROSD][258540112]utopen:6m':failed in stat OCR file/disk /vote2/storage/ocr, errno=2, os err string=No such file or directory
I was given the below explanation to why the DB node stayed down. I was hoping you guys could confirm this please as I always thought that the nodes of a RAC should automatically come back up once a problem is resolved? Any input on this would be greatly appreciated.
"When connection to the voting disks is lost, the node usually reboots. When after reboot occurs and connection is still not there, the clusterware does not start. For clusterware to restart successfully, we need connection to voting disks and data storage. This can be obtained by remount of the disks or reboot of the unix box after network connection is restored. Issues like un-plugged network connection will always require manual intervention."
Rgs,
Rob