Problem with RAC VIP .. please assist
822086Dec 8 2010 — edited Dec 8 2010Dear All,
I have a database setup using Oracle 10g. two nodes, with real IPs and two VIPS (RAC). as usual both have listeners which is using both the VIP and in the application , i am calling the lister. and based on software load balancing it is serving both the DBs. everything was working perfect.
Yesterday , our network team undergone firewall failover , at that time , for few seconds gateway wasnt reachable. Firewall failover forced two or three request time out. Due to which DB1 lost gateway reachability , i suppose and it moved its VIP to DB2 and now both VIPS are served by DB2 and DB1 is being idle now after that .
lister is serving VIPs which is active on single DB server . This issue happened exactly at same time when the firewall got failover
My question is , why it affected only DB1 , is there any mechanism through which this scenario wont happen in future. I have no clue why is this happening.
please assist.
Thanking everyone in advance.
-Parvees
Please find the log from the server below
2010-12-07 00:29:16.343: [ CRSRES][1266029] In stateChanged, ora.gia-ora-01.vip target is ONLINE
2010-12-07 00:29:16.357: [ CRSRES][1266029] ora.gia-ora-01.vip on gia-ora-01 went OFFLINE unexpectedly
2010-12-07 00:29:16.357: [ CRSRES][1266029] StopResource: setting CLI values
2010-12-07 00:29:16.385: [ CRSRES][1266029] Attempting to stop `ora.gia-ora-01.vip` on member `gia-ora-01`
2010-12-07 00:29:16.987: [ CRSRES][1266029] Stop of `ora.gia-ora-01.vip` on member `gia-ora-01` succeeded.
2010-12-07 00:29:16.988: [ CRSRES][1266029] ora.gia-ora-01.vip RESTART_COUNT=0 RESTART_ATTEMPTS=0
2010-12-07 00:29:16.992: [ CRSRES][1266029] ora.gia-ora-01.vip failed on gia-ora-01 relocating.
2010-12-07 00:29:17.331: [ CRSRES][1266029] StopResource: setting CLI values
2010-12-07 00:29:17.338: [ CRSRES][1266029] Attempting to stop `ora.gia-ora-01.LISTENER_GIA_GIA-ORA-01.lsnr` on member `gia-ora-01`
2010-12-07 00:30:10.280: [ COMMCRS][46]clsc_receive: (101cbb990) error 2
2010-12-07 00:33:03.780: [ CRSRES][1266029] Stop of `ora.gia-ora-01.LISTENER_GIA_GIA-ORA-01.lsnr` on member `gia-ora-01` succeeded.
2010-12-07 00:33:03.781: [ CRSRES][1266029] StopResource: setting CLI values
2010-12-07 00:33:03.801: [ CRSRES][1266029] Attempting to stop `ora.gia-ora-01.LISTENER_GIAKHDB_GIA-ORA-01.lsnr` on member `gia-ora-01`
2010-12-07 00:36:53.125: [ CRSRES][1266029] Stop of `ora.gia-ora-01.LISTENER_GIAKHDB_GIA-ORA-01.lsnr` on member `gia-ora-01` succeeded.
2010-12-07 00:36:53.126: [ CRSRES][1266029] StopResource: setting CLI values
2010-12-07 00:36:53.133: [ CRSRES][1266029] Attempting to stop `ora.gia-ora-01.LISTENER_ASM_GIA-ORA-01.lsnr` on member `gia-ora-01`
2010-12-07 00:40:39.455: [ CRSRES][1266029] Stop of `ora.gia-ora-01.LISTENER_ASM_GIA-ORA-01.lsnr` on member `gia-ora-01` succeeded.
2010-12-07 00:40:39.510: [ CRSRES][1266029] Attempting to start `ora.gia-ora-01.vip` on member `gia-ora-02`
2010-12-07 00:40:43.167: [ CRSRES][1266029] Start of `ora.gia-ora-01.vip` on member `gia-ora-02` succeeded.
2010-12-07 00:40:43.258: [ CRSRES][1266416] CRS-1002: Resource 'ora.gia-ora-01.vip' is already running on member 'gia-ora-02'