Hi,
I have RAC 11gR2 with 2 nodes.
One of the instances is down.
Alert log in rdbms + asm shows:
ORA-29701: unable to connect to Cluster Synchronization Service
Thu Apr 29 14:39:38 2010
MMON (ospid: 7133): terminating the instance due to error 481
I made the following checks :
crsctl check cluster
CRS-4535: Cannot communicate with Cluster Ready Services
CRS-4530: Communications failure contacting Cluster Synchronization Services daemon
CRS-4533: Event Manager is online
Then i run:
sudo crsctl start cluster
[xxx2:oracle@+ASM2 /software/oracle/dba/scripts/bin]$ sudo crsctl start cluster
CRS-2672: Attempting to start 'ora.cssd' on 'xxx2'
CRS-2672: Attempting to start 'ora.diskmon' on 'xxx2'
CRS-2676: Start of 'ora.diskmon' on 'xxx2' succeeded
CRS-2674: Start of 'ora.cssd' on 'xxx2' failed <========= Failure
CRS-2679: Attempting to clean 'ora.cssd' on 'xxx2'
CRS-2681: Clean of 'ora.cssd' on 'xxx2' succeeded
CRS-4000: Command Start failed, or completed with errors.
In the first node i see that there more two
css process that are up and running
ps -ef | grep cssd
root 9101 1 0 Apr12 ? 00:03:15 /software/oracle/GRID11gR2/bin/cssdmonitor
root 9135 1 0 Apr12 ? 00:03:24 /software/oracle/GRID11gR2/bin/cssdagent
oracle 9175 1 0 Apr12 ? 01:14:54 /software/oracle/GRID11gR2/bin/ocssd.bin
I run the follwoing command against the second node (where the instance is down):
ps -ef | grep CSS
root 11421 1 0 Apr29 ? 01:25:20 /software/oracle/GRID11gR2/bin/cssdmonitor
Thanks