node1:
export IGNORE_PREADDNODE_CHECKS=Y
Because we have warnings with NTP on the zone, we use the time in the global zone
cd $CRS_HOME/oui/bin
./addNode.sh -silent "CLUSTER_NEW_NODES={bcn1-db-oracle-106p1}" "CLUSTER_NEW_VIRTUAL_HOSTNAMES={bcn1-oracle-edreal-c1-vip2}"
Home copied to new nodes
Saving inventory on nodes (Tuesday, June 7, 2016 8:12:38 AM GMT+01:00)
. 100% Done.
node2 (the node that we are adding)
[root@bcn1-db-oracle-106p1:~]# cd /grid/oragrid_11gR2/app/grid/product/11.2.0/grid_1/
[root@bcn1-db-oracle-106p1:grid_1]# ./root.sh
/oracle/oradb_11gR2/app/oraInventory/orainstRoot.sh
ok!
[root@bcn1-db-oracle-106p1:grid_1]# ./root.sh
Installing Trace File Analyzer
Start of resource "ora.cssd" failed
CRS-2672: Attempting to start 'ora.cssdmonitor' on 'bcn1-db-oracle-106p1'
CRS-2676: Start of 'ora.cssdmonitor' on 'bcn1-db-oracle-106p1' succeeded
CRS-2672: Attempting to start 'ora.cssd' on 'bcn1-db-oracle-106p1'
CRS-2672: Attempting to start 'ora.diskmon' on 'bcn1-db-oracle-106p1'
CRS-2676: Start of 'ora.diskmon' on 'bcn1-db-oracle-106p1' succeeded
CRS-2674: Start of 'ora.cssd' on 'bcn1-db-oracle-106p1' failed
CRS-2679: Attempting to clean 'ora.cssd' on 'bcn1-db-oracle-106p1'
CRS-2681: Clean of 'ora.cssd' on 'bcn1-db-oracle-106p1' succeeded
CRS-2673: Attempting to stop 'ora.cssdmonitor' on 'bcn1-db-oracle-106p1'
CRS-2677: Stop of 'ora.cssdmonitor' on 'bcn1-db-oracle-106p1' succeeded
CRS-5804: Communication error with agent process
CRS-4000: Command Start failed, or completed with errors.
Failed to start Oracle Grid Infrastructure stack
Failed to start Cluster Synchorinisation Service in clustered mode at /grid/oragrid_11gR2/app/grid/product/11.2.0/grid_1/crs/install/crsconfig_lib.pm line 1285.
/grid/oragrid_11gR2/app/grid/product/11.2.0/grid_1/perl/bin/perl -I/grid/oragrid_11gR2/app/grid/product/11.2.0/grid_1/perl/lib -I/grid/oragrid_11gR2/app/grid/product/11.2.0/grid_1/crs/install /grid/oragrid_11gR2/app/grid/product/11.2.0/grid_1/crs/install/rootcrs.pl execution failed
# tail -100f rootcrs_bcn1-db-oracle-106p1.log
.
.
.
CRS-2676: Start of 'ora.diskmon' on 'bcn1-db-oracle-106p1' succeeded
CRS-2674: Start of 'ora.cssd' on 'bcn1-db-oracle-106p1' failed <--------------
CRS-2679: Attempting to clean 'ora.cssd' on 'bcn1-db-oracle-106p1'
CRS-2681: Clean of 'ora.cssd' on 'bcn1-db-oracle-106p1' succeeded
CRS-2673: Attempting to stop 'ora.cssdmonitor' on 'bcn1-db-oracle-106p1'
CRS-2677: Stop of 'ora.cssdmonitor' on 'bcn1-db-oracle-106p1' succeeded
CRS-5804: Communication error with agent process
CRS-4000: Command Start failed, or completed with errors.
Failed to start Oracle Grid Infrastructure stack
Failed to start Cluster Synchorinisation Service in clustered mode at /grid/oragrid_11gR2/app/grid/product/11.2.0/grid_1/crs/install/crsconfig_lib.pm line 1285. <--------------
/grid/oragrid_11gR2/app/grid/product/11.2.0/grid_1/perl/bin/perl -I/grid/oragrid_11gR2/app/grid/product/11.2.0/grid_1/perl/lib -I/grid/oragrid_11gR2/app/grid/product/11.2.0/grid_1/crs/install /grid/oragrid_11gR2/app/grid/product/11.2.0/grid_1/crs/install/rootcrs.pl execution failed
[root@bcn1-db-oracle-106p1:~]# tail -100f ocssd.log
| 2016-06-07 13:59:26.385: [ | CSSD][46]clssnmRemoveNodeInTerm: node 3, bcn1-db-oracle-106p1 terminated due to Normal Shutdown. Removing from member and connected bitmaps |
| 2016-06-07 13:59:26.385: [ | CSSD][46]################################### |
| 2016-06-07 13:59:26.385: [ | CSSD][38]clssnmvDiskPing: Writing with status 0x3, timestamp 1465300766/341559964 |
| 2016-06-07 13:59:26.385: [ | CSSD][35]clssnmvDiskPing: Writing with status 0x3, timestamp 1465300766/341559964 |
| 2016-06-07 13:59:26.385: [ | CSSD][46]clssscExit: CSSD aborting from thread clssnmRcfgMgrThread |
| 2016-06-07 13:59:26.385: [ | CSSD][32]clssnmvDiskPing: Writing with status 0x3, timestamp 1465300766/341559964 |
| 2016-06-07 13:59:26.385: [ | CSSD][46]################################### |
| 2016-06-07 13:59:26.385: [ | CSSD][46](:CSSSC00012:)clssscExit: A fatal error occurred and the CSS daemon is terminating abnormally |
.
.
| 2016-06-07 13:59:27.060: [ | CSSD][5]clssnmRemoveNodeInTerm: node 3, bcn1-db-oracle-106p1 terminated due to Normal Shutdown. Removing from member and connected bitmaps |
We follow the changes in this note but didnt work
Grid Infrastructure cssd fails to start on Solaris local containers (zones) ( Doc ID 1340694.1 )