rac load balancing question?
We have a 2node oracle rac cluster running 10.2.0.3 on RHEL4 on linux itanium. I understand oracle load balancing has both client side load balancing via tnsnames entires and also server side load balanceing which is through rac and remote_listeners parameter.
Our systems have always laod balanced connection and load very well between the 2 servers.
Today our application which is written in java and uses hibernate with ecache started having an inssue with one of the cache regions. One of my ndoes in the cluster was getting crushed by this query that was part of this cache region that apparently had blown up on the app server side. We realized someting was amiss wiht the cahing and we made the change to the ecache configuration and recycled all the jboss servers.
Afer recycling and when the cache was warming up it was still just seem to be going to one node in the cluster while the other was sitting failrly idle not doing much work.
The app.properties uses a jdbc thin connection which has both rac nodes listed and load_balance is on.
We are trying to uderstand why only one node would get crushed while the other node remained pretty idle?
Has anyone ever experienced anything similar or have any ideas why one node would be getting hit so hard while other node remains idle? This has been a very stable environemt and no recent changes to our oracle configurations.
We checked all network gear and ports and all is well no errors anywhere to speak of.
I know this is very broad but it was starnge behavior from what we are used to.
Thanks.