JVM Crash (Access Violation)
Hello Everyone,
I'm not sure this is the correct forum since my question isn't exactly about "specifications" but it was the best match i could find. If the question is inappropriate here, just tell me; I'll go and hide in shame.
I've recently run into problems with my application, causing a JVM crash because of an EXCEPTION_ACCESS_VIOLATION (0xc0000005) (full hs_err content follows below).
The problem occurs only on one specific machine (another very similar machine never had that problem; although another completely different machine might have had it, we're not sure as we don't have those logs anymore, but even if it was the same problem it wasn't as persistent as it is now). It appears to be independent of JVM version (tried multiple different 1.5 and 1.6 ones).
A little introduction to what we're trying to do: we're having a bunch of data (mostly ms-office files, pdfs, ...). We already discovered their mimetypes (if i remember correctly, through a small lookup table for the most common types and a fallback to JDIC) and have those stowed away. What we're now doing is pushing the data, into a lucene-based index, with some format normalizing (a bunch of different java libraries). It's roughly tens-of-thousands of files that we plan to do but the crash usually occurs pretty early. I'm pretty confident we don't do any JNI calls here (but there's plenty of third-party libraries involved so I can't claim to be sure about that) - the mimetypes have already been discovered, and they are kept in a database along with other file metdata so JDIC is probably not the problem.
The last hs_err.log looks like this:
#
# An unexpected error has been detected by Java Runtime Environment:
#
# EXCEPTION_ACCESS_VIOLATION (0xc0000005) at pc=0x000000000834cd94, pid=5084, tid=1904
#
# Java VM: Java HotSpot(TM) 64-Bit Server VM (10.0-b22 mixed mode windows-amd64)
# Problematic frame:
# V [jvm.dll+0x34cd94]
#
# If you would like to submit a bug report, please visit:
# http://java.sun.com/webapps/bugreport/crash.jsp
#
--------------- T H R E A D ---------------
Current thread (0x0000000006e60800): VMThread [stack: 0x0000000006fc0000,0x00000000070c0000] [id=1904]
siginfo: ExceptionCode=0xc0000005, reading address 0x00000400800c3da8
Registers:
EAX=0x0000000000000001, EBX=0x00000000001813e0, ECX=0x00000400800c3d90, EDX=0x0000000000000040
ESP=0x00000000070bf900, EBP=0x00000000d99c0000, ESI=0x00000000b2b090a0, EDI=0x00000000ab74f3e8
EIP=0x000000000834cd94, EFLAGS=0x0000000000010293
Top of Stack: (sp=0x00000000070bf900)
0x00000000070bf900: 00000000001839b0 0000000000000003
0x00000000070bf910: 000000000000120c 0000000000183a40
0x00000000070bf920: 00000000b1d28cd8 00000000ab351b70
0x00000000070bf930: 0000000000000000 00000000d95f5388
0x00000000070bf940: 00000000ab74f3c3 0000000000183a40
0x00000000070bf950: 0000000000183840 0000000000000012
0x00000000070bf960: 0000000000182190 00000000001839b0
0x00000000070bf970: 0000000000182190 000000000834d92e
0x00000000070bf980: 00000000001813e0 0000000000183a40
0x00000000070bf990: 0000000000183a40 0000000000183bf0
0x00000000070bf9a0: 0000000000183a40 000000000834bf8f
0x00000000070bf9b0: 0000000000238410 0000000000238410
0x00000000070bf9c0: 0000000000183bf0 0000000000238370
0x00000000070bf9d0: 0000000000aa6480 0000000000183a40
0x00000000070bf9e0: 0000000602fbcc13 0000000602f0f98b
0x00000000070bf9f0: 0000000602fc2463 0000000000183840
Instructions: (pc=0x000000000834cd94)
0x000000000834cd84: f5 66 66 66 90 66 66 90 66 66 66 90 48 8b 4e 08
0x000000000834cd94: 48 63 51 18 48 83 c1 10 8b c2 c1 f8 03 85 d2 7f
Stack: [0x0000000006fc0000,0x00000000070c0000], sp=0x00000000070bf900, free space=1022k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
V [jvm.dll+0x34cd94]
[error occurred during error reporting (printing native stack), id 0xc0000005]
VM_Operation (0x000000000cd9eeb0): ParallelGCFailedAllocation, mode: safepoint, requested by thread 0x0000000009476000
--------------- P R O C E S S ---------------
Java Threads: ( => current thread )
0x0000000009476000 JavaThread "Thread-12557" daemon [_thread_blocked, id=2216, stack(0x000000000cca0000,0x000000000cda0000)]
0x0000000009b73400 JavaThread "Thread-12556" daemon [_thread_blocked, id=3312, stack(0x000000000c9a0000,0x000000000caa0000)]
0x000000000867d400 JavaThread "RMI TCP Connection(idle)" daemon [_thread_blocked, id=3916, stack(0x000000000caa0000,0x000000000cba0000)]
0x000000000867c400 JavaThread "RMI TCP Connection(idle)" daemon [_thread_blocked, id=3264, stack(0x000000000cba0000,0x000000000cca0000)]
0x0000000009b74000 JavaThread "postCommitExecutor-23" daemon [_thread_blocked, id=4528, stack(0x000000000c6a0000,0x000000000c7a0000)]
0x0000000009b72c00 JavaThread "RMI RenewClean-[192.168.2.15:54007]" daemon [_thread_blocked, id=988, stack(0x000000000c4a0000,0x000000000c5a0000)]
0x0000000009475800 JavaThread "VHostExecutionContext-2" daemon [_thread_blocked, id=1032, stack(0x000000000c8a0000,0x000000000c9a0000)]
0x0000000009474c00 JavaThread "ParallelRequest-2" daemon [_thread_blocked, id=2616, stack(0x000000000c7a0000,0x000000000c8a0000)]
0x0000000009473c00 JavaThread "RMI TCP Accept-1101" daemon [_thread_in_native, id=4624, stack(0x000000000c5a0000,0x000000000c6a0000)]
0x0000000009473000 JavaThread "RMI TCP Connection(8)-10.188.0.94" daemon [_thread_blocked, id=3908, stack(0x000000000b950000,0x000000000ba50000)]
0x0000000009471c00 JavaThread "VHostExecutionContext-1" [_thread_blocked, id=1548, stack(0x000000000b650000,0x000000000b750000)]
0x0000000009471400 JavaThread "FileWatcher Thread" daemon [_thread_blocked, id=2372, stack(0x000000000c250000,0x000000000c350000)]
0x0000000009471000 JavaThread "Thread-3" daemon [_thread_blocked, id=2848, stack(0x000000000c150000,0x000000000c250000)]
0x0000000009470800 JavaThread "HSQLDB Timer @66869470" daemon [_thread_blocked, id=5016, stack(0x000000000bb50000,0x000000000bc50000)]
0x0000000009470000 JavaThread "RMI RenewClean-[192.168.2.15:53838]" daemon [_thread_blocked, id=5048, stack(0x000000000ba50000,0x000000000bb50000)]
0x0000000008c26400 JavaThread "RMI RenewClean-[192.168.2.15:1100,at.m2n.IntelligenceManagement.rmiCommunicationPlugin.impl.compressed.CompressedSocketFactory@56f0cb3c]" daemon [_thread_blocked, id=4348, stack(0x000000000b850000,0x000000000b950000)]
0x00000000095f0c00 JavaThread "RMI Scheduler(0)" daemon [_thread_blocked, id=1076, stack(0x000000000b750000,0x000000000b850000)]
0x00000000094bc400 JavaThread "GC Daemon" daemon [_thread_blocked, id=2120, stack(0x000000000b550000,0x000000000b650000)]
0x000000000978dc00 JavaThread "RMI Reaper" [_thread_blocked, id=4840, stack(0x000000000b450000,0x000000000b550000)]
0x000000000978d400 JavaThread "RMI TCP Accept-1100" daemon [_thread_in_native, id=3276, stack(0x000000000b350000,0x000000000b450000)]
0x00000000097ae800 JavaThread "RMI TCP Accept-1099" daemon [_thread_in_native, id=4712, stack(0x000000000b250000,0x000000000b350000)]
0x00000000097ae400 JavaThread "vhost scanner" daemon [_thread_blocked, id=452, stack(0x000000000b150000,0x000000000b250000)]
0x0000000008c4b800 JavaThread "RMI TCP Accept-0" daemon [_thread_in_native, id=4112, stack(0x0000000007bc0000,0x0000000007cc0000)]
0x0000000006eaf000 JavaThread "Low Memory Detector" daemon [_thread_blocked, id=3972, stack(0x00000000076c0000,0x00000000077c0000)]
0x0000000006eabc00 JavaThread "CompilerThread1" daemon [_thread_blocked, id=4676, stack(0x00000000075c0000,0x00000000076c0000)]
0x0000000006e91800 JavaThread "CompilerThread0" daemon [_thread_blocked, id=2400, stack(0x00000000074c0000,0x00000000075c0000)]
0x0000000006e8fc00 JavaThread "Attach Listener" daemon [_thread_blocked, id=5068, stack(0x00000000073c0000,0x00000000074c0000)]
0x0000000006e8f000 JavaThread "Signal Dispatcher" daemon [_thread_blocked, id=3444, stack(0x00000000072c0000,0x00000000073c0000)]
0x0000000006e65800 JavaThread "Finalizer" daemon [_thread_blocked, id=3372, stack(0x00000000071c0000,0x00000000072c0000)]
0x0000000006e64c00 JavaThread "Reference Handler" daemon [_thread_blocked, id=1716, stack(0x00000000070c0000,0x00000000071c0000)]
0x0000000000aaf800 JavaThread "main" [_thread_in_native, id=1524, stack(0x0000000002a00000,0x0000000002b00000)]
Other Threads:
=>0x0000000006e60800 VMThread [stack: 0x0000000006fc0000,0x00000000070c0000] [id=1904]
0x00000000091b4400 WatcherThread [stack: 0x0000000007db0000,0x0000000007eb0000] [id=2612]
VM state:at safepoint (normal execution)
VM Mutex/Monitor currently owned by a thread: ([mutex/lock_event])
[0x0000000000aa8590/0x0000000000000104] Threads_lock - owner thread: 0x0000000006e60800
[0x000000000017d730/0x0000000000000144] Heap_lock - owner thread: 0x0000000009476000
Heap
PSYoungGen total 1319104K, used 7936K [0x000000012beb0000, 0x000000017f400000, 0x000000017f400000)
eden space 1311168K, 0% used [0x000000012beb0000,0x000000012beb0000,0x000000017bf20000)
from space 7936K, 100% used [0x000000017bf20000,0x000000017c6e0000,0x000000017c6e0000)
to space 27200K, 0% used [0x000000017d970000,0x000000017d970000,0x000000017f400000)
PSOldGen total 1382144K, used 1378260K [0x0000000085400000, 0x00000000d99c0000, 0x000000012beb0000)
object space 1382144K, 99% used [0x0000000085400000,0x00000000d95f5388,0x00000000d99c0000)
PSPermGen total 45120K, used 31263K [0x0000000080000000, 0x0000000082c10000, 0x0000000085400000)
object space 45120K, 69% used [0x0000000080000000,0x0000000081e87fb0,0x0000000082c10000)
Dynamic libraries:
0x0000000000400000 - 0x000000000042e000 C:\Program Files\Java\jdk1.6.0_06\bin\javaw.exe
0x0000000077b80000 - 0x0000000077cfa000 C:\Windows\system32\ntdll.dll
0x0000000077a40000 - 0x0000000077b71000 C:\Windows\system32\kernel32.dll
0x000007fefe4c0000 - 0x000007fefe5bf000 C:\Windows\system32\ADVAPI32.dll
0x000007feffbd0000 - 0x000007feffd09000 C:\Windows\system32\RPCRT4.dll
0x0000000077970000 - 0x0000000077a3a000 C:\Windows\system32\USER32.dll
0x000007fefe5c0000 - 0x000007fefe621000 C:\Windows\system32\GDI32.dll
0x000007feff7c0000 - 0x000007feff7e8000 C:\Windows\system32\IMM32.DLL
0x000007feff7f0000 - 0x000007feff8f4000 C:\Windows\system32\MSCTF.dll
0x000007feff920000 - 0x000007feff9c1000 C:\Windows\system32\msvcrt.dll
0x000007feff900000 - 0x000007feff90c000 C:\Windows\system32\LPK.DLL
0x000007fefe3a0000 - 0x000007fefe43a000 C:\Windows\system32\USP10.dll
0x0000000008000000 - 0x00000000085aa000 C:\Program Files\Java\jdk1.6.0_06\jre\bin\server\jvm.dll
0x000007fefb8c0000 - 0x000007fefb8fa000 C:\Windows\system32\WINMM.dll
0x000007feff9d0000 - 0x000007feffbb0000 C:\Windows\system32\ole32.dll
0x000007fefe630000 - 0x000007fefe708000 C:\Windows\system32\OLEAUT32.dll
0x000007fefb870000 - 0x000007fefb8c0000 C:\Windows\system32\OLEACC.dll
0x0000000010000000 - 0x000000001000a000 C:\Program Files\Java\jdk1.6.0_06\jre\bin\hpi.dll
0x000007fefe240000 - 0x000007fefe248000 C:\Windows\system32\PSAPI.DLL
0x0000000000390000 - 0x000000000039e000 C:\Program Files\Java\jdk1.6.0_06\jre\bin\verify.dll
0x00000000003a0000 - 0x00000000003c7000 C:\Program Files\Java\jdk1.6.0_06\jre\bin\java.dll
0x00000000003d0000 - 0x00000000003e2000 C:\Program Files\Java\jdk1.6.0_06\jre\bin\zip.dll
0x00000000023c0000 - 0x00000000023d7000 C:\Program Files\Java\jdk1.6.0_06\jre\bin\net.dll
0x000007fefe350000 - 0x000007fefe394000 C:\Windows\system32\WS2_32.dll
0x000007feff910000 - 0x000007feff917000 C:\Windows\system32\NSI.dll
0x000007fefd6e0000 - 0x000007fefd730000 C:\Windows\system32\mswsock.dll
0x000007fefd770000 - 0x000007fefd777000 C:\Windows\System32\wship6.dll
0x000007fefd350000 - 0x000007fefd357000 C:\Windows\system32\wshtcpip.dll
0x000007fefdc50000 - 0x000007fefdc88000 C:\Windows\system32\DNSAPI.dll
0x000007fefb030000 - 0x000007fefb03b000 C:\Windows\System32\winrnr.dll
0x000007fefe2f0000 - 0x000007fefe345000 C:\Windows\system32\WLDAP32.dll
0x000007fefd140000 - 0x000007fefd153000 C:\Windows\system32\NLAapi.dll
0x000007fefd980000 - 0x000007fefd9a3000 C:\Windows\system32\IPHLPAPI.DLL
0x000007fefd930000 - 0x000007fefd976000 C:\Windows\system32\dhcpcsvc.DLL
0x000007fefe140000 - 0x000007fefe15b000 C:\Windows\system32\Secur32.dll
0x000007fefd920000 - 0x000007fefd92a000 C:\Windows\system32\WINNSI.DLL
0x000007fefd8f0000 - 0x000007fefd919000 C:\Windows\system32\dhcpcsvc6.DLL
0x000007fefb010000 - 0x000007fefb023000 C:\Windows\system32\napinsp.dll
0x000007fefaf50000 - 0x000007fefaf67000 C:\Windows\system32\pnrpnsp.dll
0x000007fefaaa0000 - 0x000007fefaaa7000 C:\Windows\system32\rasadhlp.dll
0x0000000000a70000 - 0x0000000000a7a000 C:\Program Files\Java\jdk1.6.0_06\jre\bin\management.dll
0x000007fefd450000 - 0x000007fefd493000 C:\Windows\system32\rsaenh.dll
0x000007fefe160000 - 0x000007fefe186000 C:\Windows\system32\USERENV.dll
0x0000000000a90000 - 0x0000000000a9b000 C:\Program Files\Java\jdk1.6.0_06\jre\bin\nio.dll
0x00000000023e0000 - 0x00000000023e6000 C:\Program Files\Java\jdk1.6.0_06\jre\bin\rmi.dll
VM Arguments:
jvm_args: -Xms2000m -Xmx4000m -Ddo-nolocking=true -Dcom.sun.management.jmxremote=true -XX:+HeapDumpOnOutOfMemoryError -XX:+UnlockDiagnosticVMOptions
java_command: at.m2n.IntelligenceManagement.server.Main start -configFile m2n-server.conf
Launcher Type: SUN_STANDARD
Environment Variables:
JAVA_HOME=C:\Program Files (x86)\Java\jdk1.6.0_04
PATH=C:\Program Files (x86)\Java\jre1.6.0_04\bin\client;C:\Program Files (x86)\Java\jre1.6.0_04\bin;C:\Program Files (x86)\Apache Software Foundation\Maven 1.1\bin;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Program Files (x86)\NTRU Cryptosystems\NTRU TCG Software Stack\bin\;C:\Program Files\Java\jdk1.6.0_04\bin;C:\Program Files (x86)\jEdit;C:\Program Files\MySQL\MySQL Server 5.0\bin;C:\Program Files\apache-maven-2.0.8\bin;C:\Program Files (x86)\Microsoft SQL Server\90\Tools\binn\;C:\Program Files (x86)\OpenVPN\bin
USERNAME=remote
OS=Windows_NT
PROCESSOR_IDENTIFIER=EM64T Family 6 Model 15 Stepping 11, GenuineIntel
--------------- S Y S T E M ---------------
OS: Windows Vista Build 6000
CPU:total 4 (4 cores per cpu, 1 threads per core) family 6 model 15 stepping 11, cmov, cx8, fxsr, mmx, sse, sse2, sse3, ssse3
Memory: 4k page, physical 8383896k(3137076k free), swap 16790384k(8783360k free)
vm_info: Java HotSpot(TM) 64-Bit Server VM (10.0-b22) for windows-amd64 JRE (1.6.0_06-b02), built on Mar 25 2008 01:30:15 by "java_re" with MS VC++ 8.0
time: Thu Apr 24 14:18:13 2008
elapsed time: 7214 seconds
After a bit of googling around we started the VM with the flags -XX:+UnlockDiagnosticVMOptions -XX:+VerifyBeforeGC -XX:+VerifyAfterGC to see if that would help us.
Apart from everything slowing down enormously, we got the following:
heap verification:
VerifyBeforeGC:[Verifying threads permanent tenured eden ref_proc syms strs zone dict hand C-heap ]
VerifyAfterGC:[Verifying threads permanent tenured eden Failed: 00000001435066A8 -> 0000040096D014B0
Heap
PSYoungGen total 1330048K, used 805940K [0x000000012beb0000, 0x000000017f400000, 0x000000017f400000)
eden space 1308544K, 61% used [0x000000012beb0000,0x000000015d1bd1d0,0x000000017bc90000)
from space 21504K, 0% used [0x000000017bc90000,0x000000017bc90000,0x000000017d190000)
to space 28544K, 0% used [0x000000017d820000,0x000000017d820000,0x000000017f400000)
PSOldGen total 2730688K, used 2730686K [0x0000000085400000, 0x000000012beb0000, 0x000000012beb0000)
object space 2730688K, 99% used [0x0000000085400000,0x000000012beafb88,0x000000012beb0000)
PSPermGen total 32256K, used 31967K [0x0000000080000000, 0x0000000081f80000, 0x0000000085400000)
object space 32256K, 99% used [0x0000000080000000,0x0000000081f37c28,0x0000000081f80000)
#
# An unexpected error has been detected by Java Runtime Environment:
#
# Internal Error (instanceKlass.cpp:2011), pid=992, tid=3884
# Error: guarantee(false,"boom")
#
# Java VM: Java HotSpot(TM) 64-Bit Server VM (10.0-b22 mixed mode windows-amd64)
# If you would like to submit a bug report, please visit:
# http://java.sun.com/webapps/bugreport/crash.jsp
#
--------------- T H R E A D ---------------
Current thread (0x0000000000a3ac00): VMThread [stack: 0x0000000007040000,0x0000000007140000] [id=3884]
Stack: [0x0000000007040000,0x0000000007140000]
[error occurred during error reporting (printing stack bounds), id 0xc0000005]
[error occurred during error reporting (printing native stack), id 0xc0000005]
VM_Operation (0x000000000cf3f6b0): ParallelGCFailedAllocation, mode: safepoint, requested by thread 0x000000000948d000
--------------- P R O C E S S ---------------
Java Threads: ( => current thread )
0x000000000948d000 JavaThread "Thread-442364" daemon [_thread_blocked, id=4884, stack(0x000000000ce40000,0x000000000cf40000)]
0x000000000a862800 JavaThread "Thread-442363" daemon [_thread_blocked, id=4808, stack(0x000000000cd40000,0x000000000ce40000)]
0x000000000aacb000 JavaThread "Thread-442361" daemon [_thread_blocked, id=4788, stack(0x000000000c8a0000,0x000000000c9a0000)]
0x00000000096ce400 JavaThread "RMI RenewClean-[192.168.2.15:53676]" daemon [_thread_blocked, id=3212, stack(0x000000000cc40000,0x000000000cd40000)]
0x0000000006ef1400 JavaThread "RMI RenewClean-[192.168.2.15:53151]" daemon [_thread_blocked, id=2792, stack(0x000000000cb40000,0x000000000cc40000)]
0x000000000aaca000 JavaThread "VHostExecutionContext-15" daemon [_thread_blocked, id=4412, stack(0x0000000010210000,0x0000000010310000)]
0x000000000a862400 JavaThread "VHostExecutionContext-14" daemon [_thread_blocked, id=4408, stack(0x0000000010410000,0x0000000010510000)]
0x0000000006ef2800 JavaThread "VHostExecutionContext-13" daemon [_thread_blocked, id=4388, stack(0x0000000010010000,0x0000000010110000)]
0x0000000009c67c00 JavaThread "VHostExecutionContext-12" daemon [_thread_blocked, id=4688, stack(0x000000000fc00000,0x000000000fd00000)]
0x0000000006ef2000 JavaThread "VHostExecutionContext-11" daemon [_thread_blocked, id=3092, stack(0x0000000010920000,0x0000000010a20000)]
0x00000000099e7800 JavaThread "VHostExecutionContext-10" daemon [_thread_blocked, id=3504, stack(0x0000000010620000,0x0000000010720000)]
0x00000000096cdc00 JavaThread "VHostExecutionContext-9" daemon [_thread_blocked, id=700, stack(0x0000000010520000,0x0000000010620000)]
0x00000000096cd400 JavaThread "VHostExecutionContext-8" daemon [_thread_blocked, id=2896, stack(0x000000000ff00000,0x0000000010000000)]
0x00000000099e8000 JavaThread "VHostExecutionContext-7" daemon [_thread_blocked, id=3300, stack(0x000000000fa00000,0x000000000fb00000)]
0x0000000009c66400 JavaThread "VHostExecutionContext-6" daemon [_thread_blocked, id=2280, stack(0x000000000fd00000,0x000000000fe00000)]
0x00000000099e7000 JavaThread "VHostExecutionContext-5" daemon [_thread_blocked, id=1764, stack(0x000000000f720000,0x000000000f820000)]
0x000000000a863000 JavaThread "VHostExecutionContext-4" daemon [_thread_blocked, id=1592, stack(0x000000000fb00000,0x000000000fc00000)]
0x000000000a861c00 JavaThread "VHostExecutionContext-3" daemon [_thread_blocked, id=3816, stack(0x000000000f620000,0x000000000f720000)]
0x000000000948b000 JavaThread "VHostExecutionContext-2" daemon [_thread_blocked, id=3664, stack(0x000000000f900000,0x000000000fa00000)]
0x000000000948bc00 JavaThread "ParallelRequest-2" daemon [_thread_blocked, id=3848, stack(0x000000000c7a0000,0x000000000c8a0000)]
0x000000000948b400 JavaThread "RMI TCP Connection(11)-10.188.0.94" daemon [_thread_blocked, id=3896, stack(0x000000000c6a0000,0x000000000c7a0000)]
0x000000000948a800 JavaThread "RMI TCP Accept-1101" daemon [_thread_in_native, id=3956, stack(0x000000000ca40000,0x000000000cb40000)]
0x0000000009489400 JavaThread "VHostExecutionContext-1" [_thread_blocked, id=836, stack(0x000000000c5a0000,0x000000000c6a0000)]
0x0000000009488400 JavaThread "FileWatcher Thread" daemon [_thread_blocked, id=3020, stack(0x000000000c350000,0x000000000c450000)]
0x0000000009488000 JavaThread "Thread-3" daemon [_thread_blocked, id=3852, stack(0x000000000c250000,0x000000000c350000)]
0x0000000009487800 JavaThread "HSQLDB Timer @2d0c94a7" daemon [_thread_blocked, id=1984, stack(0x000000000bc50000,0x000000000bd50000)]
0x0000000009487000 JavaThread "RMI RenewClean-[192.168.2.15:49405]" daemon [_thread_blocked, id=3352, stack(0x000000000b950000,0x000000000ba50000)]
0x000000000975a400 JavaThread "RMI RenewClean-[192.168.2.15:1100,at.m2n.IntelligenceManagement.rmiCommunicationPlugin.impl.compressed.CompressedSocketFactory@50d8a1a0]" daemon [_thread_blocked, id=3736, stack(0x000000000bb50000,0x000000000bc50000)]
0x000000000a48a800 JavaThread "RMI Scheduler(0)" daemon [_thread_blocked, id=772, stack(0x000000000ba50000,0x000000000bb50000)]
0x0000000009c96800 JavaThread "GC Daemon" daemon [_thread_blocked, id=3744, stack(0x000000000b850000,0x000000000b950000)]
0x0000000009c96400 JavaThread "RMI Reaper" [_thread_blocked, id=4044, stack(0x000000000b750000,0x000000000b850000)]
0x00000000096af400 JavaThread "RMI TCP Accept-1100" daemon [_thread_in_native, id=2220, stack(0x000000000b650000,0x000000000b750000)]
0x00000000096ae800 JavaThread "RMI TCP Accept-1099" daemon [_thread_in_native, id=3624, stack(0x000000000b550000,0x000000000b650000)]
0x0000000009855c00 JavaThread "vhost scanner" daemon [_thread_blocked, id=3976, stack(0x000000000b450000,0x000000000b550000)]
0x00000000093f8400 JavaThread "RMI TCP Accept-0" daemon [_thread_in_native, id=3100, stack(0x00000000085b0000,0x00000000086b0000)]
0x0000000006f20c00 JavaThread "Low Memory Detector" daemon [_thread_blocked, id=3792, stack(0x0000000007740000,0x0000000007840000)]
0x0000000006f1f800 JavaThread "CompilerThread1" daemon [_thread_blocked, id=2528, stack(0x0000000007640000,0x0000000007740000)]
0x0000000006f16000 JavaThread "CompilerThread0" daemon [_thread_blocked, id=476, stack(0x0000000007540000,0x0000000007640000)]
0x0000000006f0e000 JavaThread "Attach Listener" daemon [_thread_blocked, id=3652, stack(0x0000000007440000,0x0000000007540000)]
0x0000000006f0d800 JavaThread "Signal Dispatcher" daemon [_thread_blocked, id=3668, stack(0x0000000007340000,0x0000000007440000)]
0x0000000006ee4c00 JavaThread "Finalizer" daemon [_thread_blocked, id=1196, stack(0x0000000007240000,0x0000000007340000)]
0x0000000006ee3c00 JavaThread "Reference Handler" daemon [_thread_blocked, id=3684, stack(0x0000000007140000,0x0000000007240000)]
0x000000000038f800 JavaThread "main" [_thread_in_native, id=3872, stack(0x0000000002c00000,0x0000000002d00000)]
Other Threads:
=>0x0000000000a3ac00 VMThread [stack: 0x0000000007040000,0x0000000007140000] [id=3884]
0x00000000093f8c00 WatcherThread [stack: 0x000000000a380000,0x000000000a480000] [id=3568]
VM state:at safepoint (normal execution)
VM Mutex/Monitor currently owned by a thread: ([mutex/lock_event])
[0x0000000000388610/0x0000000000000104] Threads_lock - owner thread: 0x0000000000a3ac00
[0x000000000097d730/0x0000000000000144] Heap_lock - owner thread: 0x000000000948d000
Heap
PSYoungGen total 1330048K, used 805940K [0x000000012beb0000, 0x000000017f400000, 0x000000017f400000)
eden space 1308544K, 61% used [0x000000012beb0000,0x000000015d1bd1d0,0x000000017bc90000)
from space 21504K, 0% used [0x000000017bc90000,0x000000017bc90000,0x000000017d190000)
to space 28544K, 0% used [0x000000017d820000,0x000000017d820000,0x000000017f400000)
PSOldGen total 2730688K, used 2730686K [0x0000000085400000, 0x000000012beb0000, 0x000000012beb0000)
object space 2730688K, 99% used [0x0000000085400000,0x000000012beafb88,0x000000012beb0000)
PSPermGen total 32256K, used 31967K [0x0000000080000000, 0x0000000081f80000, 0x0000000085400000)
object space 32256K, 99% used [0x0000000080000000,0x0000000081f37c28,0x0000000081f80000)
Dynamic libraries:
0x0000000000400000 - 0x000000000042e000 C:\Program Files\Java\jdk1.6.0_06\bin\javaw.exe
0x0000000077b00000 - 0x0000000077c7a000 C:\Windows\system32\ntdll.dll
0x00000000779c0000 - 0x0000000077af1000 C:\Windows\system32\kernel32.dll
0x000007feff230000 - 0x000007feff32f000 C:\Windows\system32\ADVAPI32.dll
0x000007feff3a0000 - 0x000007feff4d9000 C:\Windows\system32\RPCRT4.dll
0x00000000778f0000 - 0x00000000779ba000 C:\Windows\system32\USER32.dll
0x000007feff330000 - 0x000007feff391000 C:\Windows\system32\GDI32.dll
0x000007feffbd0000 - 0x000007feffbf8000 C:\Windows\system32\IMM32.DLL
0x000007feffac0000 - 0x000007feffbc4000 C:\Windows\system32\MSCTF.dll
0x000007feffcf0000 - 0x000007feffd91000 C:\Windows\system32\msvcrt.dll
0x000007feffab0000 - 0x000007feffabc000 C:\Windows\system32\LPK.DLL
0x000007feff6e0000 - 0x000007feff77a000 C:\Windows\system32\USP10.dll
0x0000000008000000 - 0x00000000085aa000 C:\Program Files\Java\jdk1.6.0_06\jre\bin\server\jvm.dll
0x000007fefbed0000 - 0x000007fefbf0a000 C:\Windows\system32\WINMM.dll
0x000007feff500000 - 0x000007feff6e0000 C:\Windows\system32\ole32.dll
0x000007fefeeb0000 - 0x000007fefef88000 C:\Windows\system32\OLEAUT32.dll
0x000007fefbe00000 - 0x000007fefbe50000 C:\Windows\system32\OLEACC.dll
0x0000000010000000 - 0x000000001000a000 C:\Program Files\Java\jdk1.6.0_06\jre\bin\hpi.dll
0x000007fefe260000 - 0x000007fefe268000 C:\Windows\system32\PSAPI.DLL
0x00000000001a0000 - 0x00000000001ae000 C:\Program Files\Java\jdk1.6.0_06\jre\bin\verify.dll
0x00000000002b0000 - 0x00000000002d7000 C:\Program Files\Java\jdk1.6.0_06\jre\bin\java.dll
0x00000000002e0000 - 0x00000000002f2000 C:\Program Files\Java\jdk1.6.0_06\jre\bin\zip.dll
0x0000000000300000 - 0x0000000000317000 C:\Program Files\Java\jdk1.6.0_06\jre\bin\net.dll
0x000007feff780000 - 0x000007feff7c4000 C:\Windows\system32\WS2_32.dll
0x000007feff220000 - 0x000007feff227000 C:\Windows\system32\NSI.dll
0x000007fefd660000 - 0x000007fefd6b0000 C:\Windows\system32\mswsock.dll
0x000007fefd6f0000 - 0x000007fefd6f7000 C:\Windows\System32\wship6.dll
0x000007fefd290000 - 0x000007fefd297000 C:\Windows\system32\wshtcpip.dll
0x000007fefdbd0000 - 0x000007fefdc08000 C:\Windows\system32\DNSAPI.dll
0x000007fefb0c0000 - 0x000007fefb0cb000 C:\Windows\System32\winrnr.dll
0x000007feffda0000 - 0x000007feffdf5000 C:\Windows\system32\WLDAP32.dll
0x000007fefcee0000 - 0x000007fefcef3000 C:\Windows\system32\NLAapi.dll
0x000007fefd900000 - 0x000007fefd923000 C:\Windows\system32\IPHLPAPI.DLL
0x000007fefd8b0000 - 0x000007fefd8f6000 C:\Windows\system32\dhcpcsvc.DLL
0x000007fefe0c0000 - 0x000007fefe0db000 C:\Windows\system32\Secur32.dll
0x000007fefd8a0000 - 0x000007fefd8aa000 C:\Windows\system32\WINNSI.DLL
0x000007fefd870000 - 0x000007fefd899000 C:\Windows\system32\dhcpcsvc6.DLL
0x000007fefaef0000 - 0x000007fefaf03000 C:\Windows\system32\napinsp.dll
0x000007fefaed0000 - 0x000007fefaee7000 C:\Windows\system32\pnrpnsp.dll
0x000007fefaa20000 - 0x000007fefaa27000 C:\Windows\system32\rasadhlp.dll
0x0000000000360000 - 0x000000000036a000 C:\Program Files\Java\jdk1.6.0_06\jre\bin\management.dll
0x000007fefd390000 - 0x000007fefd3d3000 C:\Windows\system32\rsaenh.dll
0x000007fefe0e0000 - 0x000007fefe106000 C:\Windows\system32\USERENV.dll
0x0000000000370000 - 0x000000000037b000 C:\Program Files\Java\jdk1.6.0_06\jre\bin\nio.dll
0x0000000000390000 - 0x0000000000396000 C:\Program Files\Java\jdk1.6.0_06\jre\bin\rmi.dll
VM Arguments:
jvm_args: -Xms2000m -Xmx4000m -Ddo-nolocking=true -Dcom.sun.management.jmxremote=true -XX:+HeapDumpOnOutOfMemoryError -XX:+UnlockDiagnosticVMOptions -XX:+VerifyBeforeGC -XX:+VerifyAfterGC
java_command: at.m2n.IntelligenceManagement.server.Main start -configFile m2n-server.conf
Launcher Type: SUN_STANDARD
Environment Variables:
JAVA_HOME=C:\Program Files (x86)\Java\jdk1.6.0_04
PATH=C:\Program Files (x86)\Java\jre1.6.0_04\bin\client;C:\Program Files (x86)\Java\jre1.6.0_04\bin;C:\Program Files\Firefox;C:\Program Files (x86)\Apache Software Foundation\Maven 1.1\bin;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Program Files (x86)\NTRU Cryptosystems\NTRU TCG Software Stack\bin\;C:\Program Files\Java\jdk1.6.0_04\bin;C:\Program Files (x86)\jEdit;C:\Program Files\MySQL\MySQL Server 5.0\bin;C:\Program Files\apache-maven-2.0.8\bin;C:\Program Files (x86)\Microsoft SQL Server\90\Tools\binn\;C:\Program Files (x86)\Subversion\bin;C:\Program Files (x86)\OpenVPN\bin
USERNAME=remote
OS=Windows_NT
PROCESSOR_IDENTIFIER=EM64T Family 6 Model 15 Stepping 11, GenuineIntel
--------------- S Y S T E M ---------------
OS: Windows Vista Build 6000
CPU:total 4 (4 cores per cpu, 1 threads per core) family 6 model 15 stepping 11, cmov, cx8, fxsr, mmx, sse, sse2, sse3, ssse3
Memory: 4k page, physical 8383896k(1352440k free), swap 16806000k(8217376k free)
vm_info: Java HotSpot(TM) 64-Bit Server VM (10.0-b22) for windows-amd64 JRE (1.6.0_06-b02), built on Mar 25 2008 01:30:15 by "java_re" with MS VC++ 8.0
time: Sat May 10 15:28:18 2008
elapsed time: 183804 seconds
I can also provide more logfiles if needed. The heap which looks pretty full in the two above doesn't appear to be the problem either because i've seen the same problem with none of the heap areas above 70% used.
I'd be happy about every hint because honestly I don't have the faintest idea what to do about this.
Thanks in advance,
Wolfgang Groi�