Java HotSpot Virtual Machine

Hi,

We are having an issue where a Java process (running on Solaris 10) is using higher than expected native memory. To clarify, by native memory I mean at the operating system process level - this is NOT a Java heap issue. We have our JVM (Sun, 1.5.0_u18) configured with Xmx of 1024M and 768M perm gen, so we expect total Solaris process memory usage of <2Gb. However, according to PRSTAT we can see total process memory size is >3Gb and does go higher on occassion.

We want to try and track down what is causing this excessive memory usage, but are unsure how to go about doing this. We have used the PMAP command, which has confirmed the results reported by PRSTAT, and has also indicated that the disparity between the expected and actual memory usage appears to be primarily due to Anonymous memory. I know there are number of reasons a JVM can utilize memory outside the defined heap space, and I'm looking for some tips on how to identify which of these might be the culprit in this case.

The JVM in question is running a WebLogic Server (v9.2.3) which in turn is hosting in excess of 50 J2EE applications components which have been created by our organisation. These components in turn each may rely on a large number of different 3rd party libraries - Spring framework, JAXB, Xerces to name but a few. This makes identifying the problem from a code perspective very difficult - there are probably several million lines of code in total on the JVM! Therefore, if we can narrow thing down with OS tools or similar that would be great. One thing I can say is that I don't believe the issue is in WebLogic itself, as a JVM with WebLogic running but no apps deployed behaves fairly much as expected in terms of memory usage.

I know DTrace is one option that may assist us but we have little experience with this and I believe its a fairly complex tool. Additional, this JVM represents a Production environment, so the less invasive we can be, the better!

We have previously encountered a similar issue due to a Java bug (see http://bugs.sun.com/view_bug.do?bug_id=4797189) but we don't believe (or at least don't have any evidence to indicate) that the current problem is the same or related.

Any assistance would be greatly appreciated!

Kind Regards,

Adrian