Database Software

Hi ,

I my 11gr2 - 2node RAC senarion my one of the node get rebooted with Network timeout error but when i go through the OSWatcher private interconnect monitoring logs, there i am able to see that traceroute between the 2 node during that time is working properly.

Not able to understand that if the traceroute is working properly than why the node is getting rebooted with Network timeout error .

Please help

Regards


Please find below the cluster alertlog :

2013-03-11 22:11:44.208
[cssd(22589)]CRS-1611:Network communication with node kwhq2 (2) missing for 75% of timeout interval. Removal of this node from cluster in 6.535 seconds
2013-03-11 22:11:48.210
[cssd(22589)]CRS-1610:Network communication with node kwhq2 (2) missing for 90% of timeout interval. Removal of this node from cluster in 2.522 seconds
2013-03-11 22:11:53.630
[cssd(22589)]CRS-1607:Node kwhq2 is being evicted in cluster incarnation 235735939; details at (:CSSNM00007:) in /u01/app/11.2.0/grid/log/kwhq1/cssd/ocssd.log.


OSWatcher log for privateinterconnect for that same period : -

zzz ***Mon Mar 11 22:10:35 AST 2013
traceroute: Warning: Multiple interfaces found; using 192.168.40.1 @ vnickwhqhb0
traceroute to 192.168.40.2 (192.168.40.2), 30 hops max, 40 byte packets
1 kwhq2-priv (192.168.40.2) 0.446 ms 0.343 ms 0.294 ms
zzz ***Mon Mar 11 22:10:50 AST 2013
traceroute: Warning: Multiple interfaces found; using 192.168.40.1 @ vnickwhqhb0
traceroute to 192.168.40.2 (192.168.40.2), 30 hops max, 40 byte packets
1 kwhq2-priv (192.168.40.2) 0.326 ms 0.258 ms 0.136 ms
zzz ***Mon Mar 11 22:11:05 AST 2013
traceroute: Warning: Multiple interfaces found; using 192.168.40.1 @ vnickwhqhb0
traceroute to 192.168.40.2 (192.168.40.2), 30 hops max, 40 byte packets
1 kwhq2-priv (192.168.40.2) 0.294 ms 0.232 ms 0.125 ms
zzz ***Mon Mar 11 22:11:21 AST 2013
traceroute: Warning: Multiple interfaces found; using 192.168.40.1 @ vnickwhqhb0
traceroute to 192.168.40.2 (192.168.40.2), 30 hops max, 40 byte packets
1 kwhq2-priv (192.168.40.2) 0.348 ms 0.166 ms 0.142 ms
zzz ***Mon Mar 11 22:11:36 AST 2013
traceroute: Warning: Multiple interfaces found; using 192.168.40.1 @ vnickwhqhb0
traceroute to 192.168.40.2 (192.168.40.2), 30 hops max, 40 byte packets
1 kwhq2-priv (192.168.40.2) 0.275 ms 0.321 ms 0.147 ms
zzz ***Mon Mar 11 22:11:51 AST 2013
traceroute: Warning: Multiple interfaces found; using 192.168.40.1 @ vnickwhqhb0
traceroute to 192.168.40.2 (192.168.40.2), 30 hops max, 40 byte packets
1 kwhq2-priv (192.168.40.2) 0.325 ms 0.214 ms 0.125 ms
zzz ***Mon Mar 11 22:12:06 AST 2013
traceroute: Warning: Multiple interfaces found; using 192.168.40.1 @ vnickwhqhb0
traceroute to 192.168.40.2 (192.168.40.2), 30 hops max, 40 byte packets
1 kwhq2-priv (192.168.40.2) 0.213 ms 0.239 ms 0.122 ms