Hi,
In a production environment encountered a memory leak problem after running over 200 days, and i found KeepAliveCache.java will produce memory leaks if it create native thread failed.
I think if keepAliveTimer.start() failed, java.lang.Thread object was not destory and can not collection by GC.
java.security.AccessController.doPrivileged(
new java.security.PrivilegedAction<Void>() {
public Void run() {
// We want to create the Keep-Alive-Timer in the
// system threadgroup
ThreadGroup grp = Thread.currentThread().getThreadGroup();
ThreadGroup parent = null;
while ((parent = grp.getParent()) != null) {
grp = parent;
}
keepAliveTimer = new Thread(grp, cache, "Keep-Alive-Timer");
keepAliveTimer.setDaemon(true);
keepAliveTimer.setPriority(Thread.MAX_PRIORITY - 2);
// Set the context class loader to null in order to avoid
// keeping a strong reference to an application classloader.
keepAliveTimer.setContextClassLoader(null);
keepAliveTimer.start();
return null;
}
});
we found the top 10 objects used memory are as follows
num #instances #bytes class name
----------------------------------------------
1: 2287176 256163712 java.lang.Thread
2: 3200649 197768504 [C
3: 2378720 190314944 [Ljava.lang.ThreadLocal$ThreadLocalMap$Entry;
4: 182033 90707352 [Ljava.lang.Object;
5: 2382405 76236960 java.lang.ThreadLocal$ThreadLocalMap$Entry
6: 2378720 57089280 java.lang.ThreadLocal$ThreadLocalMap
7: 2286842 54884208 java.security.AccessControlContext
8: 212233 28031904 <constMethodKlass>
9: 902205 21652920 java.lang.String
10: 2424218 19393744 java.lang.Object
But there are not so many threads in the thread dump, and we found many error log as following:
07:43:21.748 ERROR WebContainerAdvertiserThread ? **.webcontainer.advertiser.startup.WebContainerAdvertiserThread advertise WebContainerAdvertiserThread.java 105
Exception Occured when advertising WebContainer Details
java.lang.OutOfMemoryError: unable to create new native thread
at java.lang.Thread.start0(Native Method)
at java.lang.Thread.start(Thread.java:640)
at sun.net.www.http.KeepAliveCache$1.run(KeepAliveCache.java:89)
at java.security.AccessController.doPrivileged(Native Method)
at sun.net.www.http.KeepAliveCache.put(KeepAliveCache.java:75)
at sun.net.www.http.HttpClient.putInKeepAliveCache(HttpClient.java:370)
at sun.net.www.http.HttpClient.finished(HttpClient.java:358)
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1393)
at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:379)
Java(TM) SE Runtime Environment (build 1.6.0_25-b06)
SUSE Linux Enterprise Server 10.3
Regards