Hi guys,
I'm trying to figure out what is causing the native memory leak in JVM (hotspot 6.0_43-b01 32 bits in Solaris 11 Sparc 64 bits - currently we have a JVM 32-bit architecture but we are working to move).
# There is insufficient memory for the Java Runtime Environment to continue.
# Native memory allocation (malloc) failed to allocate 909712 bytes for Chunk::new
# Possible reasons:
# The system is out of physical RAM or swap space
# In 32 bit mode, the process size limit was hit
# Possible solutions:
# Reduce memory load on the system
# Increase physical memory or swap space
# Check if swap backing store is full
# Use 64 bit Java on a 64 bit OS
# Decrease Java heap size (-Xmx/-Xms)
# Decrease number of Java threads
# Decrease Java thread stack sizes (-Xss)
# Set larger code cache with -XX:ReservedCodeCacheSize=
# This output file may be truncated or incomplete.
#
# Out of Memory Error (allocation.cpp:317), pid=26465, tid=25
#
# JRE version: 6.0_43-b01
# Java VM: Java HotSpot(TM) Server VM (20.14-b01 mixed mode solaris-sparc )
--------------- T H R E A D ---------------
Current thread (0x006aa400): JavaThread "C2 CompilerThread1" daemon [_thread_in_native, id=25, stack(0x302e0000,0x30360000)]
Stack: [0x302e0000,0x30360000], sp=0x3035d720, free space=501k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
V [libjvm.so+0x8cd0cc]
Following an Oracle procedure using libumem I was able to generate a log (from core file) with umausers and umausers oversized.
It's possible to see more than 5300 allocations of 32kb , most of them with the following stack trace:
150a960 ALLC 1e62000 1e6a008 32776
42 a311e4b73569f2
libumem.so.1`vmem_seg_alloc+0x170
libumem.so.1`vmem_xalloc+0x5c0
libumem.so.1`vmem_alloc+0x1ac
libumem.so.1`umem_alloc+0xe4
libumem.so.1`malloc+0x28
libjvm.so`void*os::malloc+0x24
libjvm.so`Unsafe_AllocateMemory+0x214
0xfc00d700
0xfc00d6ac
0xfc0059d0
0xfc005ab0
0xfc005940
0xfc005940
0xfc005940
0xfc005e60
0xfc005940
0xfc005ab0
0xfc005fd0
0xfc005ab0
0xfc005ab0
I know that Unsafe.allocateMemory() is a wrapper around os::malloc which doesn't care about any memory limits imposed by the VM and is allocating in native memory.
My big issue is to understand what is calling this method (stack trace below libjvm.so`Unsafe_AllocateMemory+0x214 is not human readable) and If it's supposed to have GC working on this memory structure.
Do you have any idea how to move forward with the troubleshooting in this issue?
Thanks