Skip to Main Content

Database Software

Announcement

For appeals, questions and feedback about Oracle Forums, please email oracle-forums-moderators_us@oracle.com. Technical questions should be asked in the appropriate category. Thank you!

CRS stack seems to be down but DB instance is up in Node1 : Two questions

C. BoutetNov 11 2015 — edited Nov 12 2015

GI version: 11.2.0.4

DB Version: 11.2.0.4

OS : Oracle Linux 6.5

We have a 2-node RAC DB and today early morning, in Node1, there was some issue accessing OCR_VOTE diskgroup and the CRS went down.

But the DB seems to be running fine. We came to know this only from OEM alerts saying

"[crsd(27504)]CRS-2765:Resource 'ora.OCR_VOTE_DG.dg' has failed on server 'wmsthprd198'."

-- From Node2

"[ohasd(25660)]CRS-2769:Unable to failover resource 'ora.crsd'. See /grid_infra/product/11.2.0/log/wmsthprd199/alertwmsthprd199.log for details. "

In the morning, when we reached office, We found the following in the grid alert log.

[crsd(27504)]CRS-2765:Resource 'ora.OCR_VOTE_DG.dg' has failed on server 'wmsthprd198'.

2015-11-11 05:36:50.643:

[crsd(27504)]CRS-2878:Failed to restart resource 'ora.OCR_VOTE_DG.dg'

2015-11-11 05:36:50.651:

So, We had to manually mount the OCR_VOTE_DG diskgroup in Node1. But, the CRS seems to be down. A restart of the Node1 will most likely fix the issue. But, I have 2 questions

Question1 (a theoretical one)

CRS is not strictly mandatory to be up in a Node for the DB to functioning. Right ? Of course, I cannot do anythithing with srvctl in this node.But DB instance seems to fine.

Question2.

A restart of the CRS in this node will most likely fix this issue. But the next available maintenance window is only after 36 hours. Is it OK to leave the cluster in this state in this Node ?

[root@wmsthprd198 ~]# /grid_infra/product/11.2.0/bin/crsctl check crs

CRS-4638: Oracle High Availability Services is online

CRS-4535: Cannot communicate with Cluster Ready Services

CRS-4529: Cluster Synchronization Services is online

CRS-4533: Event Manager is online

[root@wmsthprd198 ~]# /grid_infra/product/11.2.0/bin/crsctl status resource -t

CRS-4535: Cannot communicate with Cluster Ready Services

CRS-4000: Command Status failed, or completed with errors.

[root@wmsthprd198 ~]# su - oracle

[oracle@wmsthprd198 ~]$

[oracle@wmsthprd198 ~]$ . oraenv

ORACLE_SID = [oracle] ? BRCSPRD1

The Oracle base has been set to /dboracle/product

[oracle@wmsthprd198 ~]$


[oracle@wmsthprd198 ~]$ srvctl status database -d BRCSPRD1

PRCD-1027 : Failed to retrieve database BRCSPRD1

PRCR-1115 : Failed to find entities of type resource that match filters ((NAME == ora.brcsprd1.db) && (TYPE == ora.database.type)) and contain attributes VERSION,ORACLE_HOME,DATABASE_TYPE

Cannot communicate with crsd

[oracle@wmsthprd198 ~]$

[oracle@wmsthprd198 ~]$ sqlplus / as sysdba

SQL*Plus: Release 11.2.0.4.0 Production on Wed Nov 11 14:21:00 2015

Copyright (c) 1982, 2013, Oracle.  All rights reserved.


Connected to:

Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production

With the Partitioning, Real Application Clusters, Automatic Storage Management, OLAP,

Data Mining and Real Application Testing options

SQL> select INSTANCE_NAME, status from gv$instance;

INSTANCE_NAME    STATUS

---------------- ------------

BRCSPRD1         OPEN

BRCSPRD2         OPEN

In Node2, everything is fine ie. CRS stack is up

# /grid_infra/product/11.2.0/bin/crsctl check crs

CRS-4638: Oracle High Availability Services is online

CRS-4537: Cluster Ready Services is online

CRS-4529: Cluster Synchronization Services is online

CRS-4533: Event Manager is online

#

Comments
Locked Post
New comments cannot be posted to this locked post.
Post Details
Locked on Dec 10 2015
Added on Nov 11 2015
4 comments
3,559 views