JVM process leaking memory, but Java Heap is fine...Very difficult problem
843829Feb 6 2009 — edited Feb 11 2009Our JVM (java process) is leaking memory at a rapid pace. Meaning, the java process continues to grow and grow in virtual and resident memory until Linux kills it via the OOM killer. The leak is relatively slow. There is no huge jump. It takes 10-14 days before it gets to 6 GB virtual. It's insane.
However, from a Java memory perspective, the heap is fine. Garbage collection is working like a champ. No Java out of memory errors. Not even close.
I tried to thik of what non-heap memory could be causing this. Crazy amount of threads was my first thought, but that isn't occurring from a Java or OS perspective.
So, my questions are:
- What could we possibly have done in our code to cause this? We know of no major changes to our code preceding this problem, and you would think that if our code was leaking memory a Java OOM would occur.
- From an OS perspective, what changes could cause this? Our Linux support team swears no major changes were made at the OS level.
- Finally, how can we figure out what the heck is causing the problem? We've profiled our code and don't see any issues. What should we try next? A JVM upgrade is all we can think of, but it doesn't makes sense that this would just start happening for no reason.
Java and Linux version info:
java version "1.5.0_09"
Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_09-b01)
Java HotSpot(TM) 64-Bit Server VM (build 1.5.0_09-b01, mixed mode)
Red Hat Enterprise Linux AS release 4 (Nahant Update 4)
Any help would be appreciated beyond words.
Thanks.
Edited by: bigdaddy on Feb 6, 2009 3:54 PM
Edited by: bigdaddy on Feb 6, 2009 4:18 PM