Java HotSpot Virtual Machine

Hi,
I've spent days and days trying to figure this one out. Apparently this crash happen pretty randomly and I haven't been able to reproduce it on a regular basis. The crash may have something to do with our recent upgrade from JDK 1.3 to JDK 1.5.0_06-b05 mixed mode. Further I have yet to experience the problem on Linux, so it may turn out to be a Windows spesific problem.

Basically the system consist of a Java layer, a JNI "glue layer" developed in C and our main application in plain native C. The C layer is compiled with Borland C++ compiler version 5.5.

I have also tried to create a dump file using userdump.exe and tried to analyze it in Windbg. The result is posted after the crash dump,.

I will greatly appreciate any input that can lead me closer to identify the bug.Please ask if there is more information I should provide.

Regards,
Rolv

Here is the complete crash dump:
#
# An unexpected error has been detected by HotSpot Virtual Machine:
#
# EXCEPTION_FLT_STACK_CHECK (0xc0000092) at pc=0x008741c6, pid=4752, tid=6268
#
# Java VM: Java HotSpot(TM) Server VM (1.5.0_06-b05 mixed mode)
# Problematic frame:
# v ~DeoptimizationBlob
#

--------------- T H R E A D ---------------

Current thread (0x102aaab0): JavaThread "MultiDispatcherWorker" daemon [_thread_in_Java, id=6268]

siginfo: ExceptionCode=0xc0000092, ExceptionInformation=0x00000000 0x11cff7cc

Registers:
EAX=0x0da8ae20, EBX=0x008707c0, ECX=0x008741b1, EDX=0x009e3428
ESP=0x11cff760, EBP=0x0da62dd0, ESI=0x06917cf8, EDI=0x00000001
EIP=0x008741c6, EFLAGS=0x00010206

Top of Stack: (sp=0x11cff760)
0x11cff760: ffff1372 ffff0020 ffffffff 00857b37
0x11cff770: 0051001b 11cff774 ffff0023 00690076
0x11cff780: 00650063 0022ffff 00730020 ffff0064
0x11cff790: 00530050 00720065 0000ffff 80000000
0x11cff7a0: 400cbe30 47ae1800 efdffae1 0000400e
0x11cff7b0: 00000000 40018000 00000000 80000000
0x11cff7c0: 0000c007 41200000 401b8c1f 102aaab0
0x11cff7d0: 02f94ca1 00857b20 0da8ae20 00000000

Instructions: (pc=0x008741c6)
0x008741b6: ec 78 83 ec 6c dd 34 24 9b dd 24 24 dd 5c 24 6c
0x008741c6: dd 5c 24 74 dd 5c 24 7c dd 9c 24 84 00 00 00 dd


Stack: [0x11b00000,0x11d00000), sp=0x11cff760, free space=2045k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
v ~DeoptimizationBlob


--------------- P R O C E S S ---------------

Java Threads: ( => current thread )
0x0f218c88 JavaThread "DestroyJavaVM" [_thread_blocked, id=4744]
=>0x102aaab0 JavaThread "MultiDispatcherWorker" daemon [_thread_in_Java, id=6268]
0x0f1557f0 JavaThread "PROD7777_O_OKO02" [_thread_in_native, id=6272]
0x10268ae8 JavaThread "MultiProcess_O_Oko02" [_thread_in_native, id=5660]
0x0f07c420 JavaThread "Low Memory Detector" daemon [_thread_blocked, id=4684]
0x0f07b1b8 JavaThread "CompilerThread1" daemon [_thread_blocked, id=4700]
0x0f07a348 JavaThread "CompilerThread0" daemon [_thread_blocked, id=4704]
0x0f079220 JavaThread "AdapterThread" daemon [_thread_blocked, id=4708]
0x0f078578 JavaThread "Signal Dispatcher" daemon [_thread_blocked, id=4712]
0x0084fb80 JavaThread "Finalizer" daemon [_thread_blocked, id=4716]
0x0084eee0 JavaThread "Reference Handler" daemon [_thread_blocked, id=5176]

Other Threads:
0x0084cca8 VMThread [id=4728]
0x0f07d6a0 WatcherThread [id=4680]

VM state:not at safepoint (normal execution)

VM Mutex/Monitor currently owned by a thread: None

Heap
PSYoungGen total 14272K, used 477K [0x0da20000, 0x0e830000, 0x0e850000)
eden
[error occurred during error reporting, step 190, id 0xc0000092]

Dynamic libraries:
0x00400000 - 0x0040c000 C:\Program Files\Java\jdk1.5.0_06\bin\java.exe
0x77f40000 - 0x77ffa000 C:\WINDOWS\system32\ntdll.dll
0x77e40000 - 0x77f34000 C:\WINDOWS\system32\kernel32.dll
0x77da0000 - 0x77e30000 C:\WINDOWS\system32\ADVAPI32.dll
0x77c50000 - 0x77cf5000 C:\WINDOWS\system32\RPCRT4.dll
0x77ba0000 - 0x77bf4000 C:\WINDOWS\system32\MSVCRT.dll
0x6d880000 - 0x6dc31000 C:\Program Files\Java\jdk1.5.0_06\jre\bin\server\jvm.dll
0x77d00000 - 0x77d8f000 C:\WINDOWS\system32\USER32.dll
0x77c00000 - 0x77c44000 C:\WINDOWS\system32\GDI32.dll
0x76aa0000 - 0x76acc000 C:\WINDOWS\system32\WINMM.dll
0x71bc0000 - 0x71bc8000 C:\WINDOWS\system32\rdpsnd.dll
0x76260000 - 0x76270000 C:\WINDOWS\system32\WINSTA.dll
0x71c40000 - 0x71c93000 C:\WINDOWS\system32\NETAPI32.dll
0x76b70000 - 0x76b7b000 C:\WINDOWS\system32\PSAPI.DLL
0x6d2f0000 - 0x6d2f8000 C:\Program Files\Java\jdk1.5.0_06\jre\bin\hpi.dll
0x6d6b0000 - 0x6d6bc000 C:\Program Files\Java\jdk1.5.0_06\jre\bin\verify.dll
0x6d370000 - 0x6d38d000 C:\Program Files\Java\jdk1.5.0_06\jre\bin\java.dll
0x6d6d0000 - 0x6d6df000 C:\Program Files\Java\jdk1.5.0_06\jre\bin\zip.dll
0x10480000 - 0x104f8000 E:\multiplus\standard\mas_v1g_3\muex\multiglue.dll
0x1c000000 - 0x1c013000 C:\IBM\WebSphere MQ\Java\lib\mqjbnd05.dll
0x4f520000 - 0x4f52a000 C:\IBM\WebSphere MQ\bin\MQM.dll
0x4e850000 - 0x4e917000 C:\IBM\WebSphere MQ\bin\AMQXCS2.dll
0x4e960000 - 0x4e970000 C:\IBM\WebSphere MQ\bin\amqxcsn.DLL
0x4f0f0000 - 0x4f148000 C:\IBM\WebSphere MQ\bin\AMQZST.dll
0x4ef50000 - 0x4ef5b000 C:\IBM\WebSphere MQ\bin\AMQZC.dll
0x4e790000 - 0x4e7a3000 C:\IBM\WebSphere MQ\bin\AMQVWAA2.dll
0x4f6b0000 - 0x4f6b9000 C:\IBM\WebSphere MQ\bin\MQMVXD.dll
0x4e250000 - 0x4e283000 C:\IBM\WebSphere MQ\bin\amqmtmgr.dll
0x77160000 - 0x77286000 C:\WINDOWS\system32\ole32.dll
0x770e0000 - 0x7715d000 C:\WINDOWS\system32\OLEAUT32.dll
0x4f170000 - 0x4f17d000 C:\IBM\WebSphere MQ\bin\AMQZTM.dll
0x4f060000 - 0x4f07a000 C:\IBM\WebSphere MQ\bin\amqzsai.dll
0x76f50000 - 0x76f63000 C:\WINDOWS\system32\Secur32.dll
0x6d530000 - 0x6d543000 C:\Program Files\Java\jdk1.5.0_06\jre\bin\net.dll
0x71c00000 - 0x71c18000 C:\WINDOWS\system32\WS2_32.dll
0x71bf0000 - 0x71bf8000 C:\WINDOWS\system32\WS2HELP.dll
0x71b20000 - 0x71b63000 C:\WINDOWS\System32\mswsock.dll
0x76ed0000 - 0x76ef7000 C:\WINDOWS\system32\DNSAPI.dll
0x76f70000 - 0x76f77000 C:\WINDOWS\System32\winrnr.dll
0x76f10000 - 0x76f3f000 C:\WINDOWS\system32\WLDAP32.dll
0x76f80000 - 0x76f85000 C:\WINDOWS\system32\rasadhlp.dll
0x71ae0000 - 0x71ae8000 C:\WINDOWS\System32\wshtcpip.dll
0x62f00000 - 0x62f13000 E:\oracle\product\10.2.0\Client_2\BIN\ocijdbc10.dll
0x11190000 - 0x111e7000 E:\Oracle\product\10.2.0\Client_2\bin\OCI.dll
0x7c340000 - 0x7c396000 E:\Oracle\product\10.2.0\Client_2\bin\MSVCR71.dll
0x61c20000 - 0x61e72000 E:\oracle\product\10.2.0\Client_2\bin\OraClient10.Dll
0x60870000 - 0x60953000 E:\oracle\product\10.2.0\Client_2\bin\oracore10.dll
0x60a80000 - 0x60b44000 E:\oracle\product\10.2.0\Client_2\bin\oranls10.dll
0x63690000 - 0x636a8000 E:\oracle\product\10.2.0\Client_2\bin\oraunls10.dll
0x60eb0000 - 0x60eb7000 E:\oracle\product\10.2.0\Client_2\bin\orauts.dll
0x636b0000 - 0x636b6000 E:\oracle\product\10.2.0\Client_2\bin\oravsn10.dll
0x60fa0000 - 0x61092000 E:\oracle\product\10.2.0\Client_2\bin\oracommon10.dll
0x60300000 - 0x60857000 E:\oracle\product\10.2.0\Client_2\bin\orageneric10.dll
0x63430000 - 0x63457000 E:\oracle\product\10.2.0\Client_2\bin\orasnls10.dll
0x11210000 - 0x11370000 E:\oracle\product\10.2.0\Client_2\bin\oraxml10.dll
0x60020000 - 0x60030000 C:\WINDOWS\system32\MSVCIRT.dll
0x60960000 - 0x60a6d000 E:\oracle\product\10.2.0\Client_2\bin\oran10.dll
0x62740000 - 0x6277d000 E:\oracle\product\10.2.0\Client_2\bin\oranl10.dll
0x62790000 - 0x627a7000 E:\oracle\product\10.2.0\Client_2\bin\oranldap10.dll
0x627f0000 - 0x628f9000 E:\oracle\product\10.2.0\Client_2\bin\orannzsbb10.dll
0x62530000 - 0x62583000 E:\oracle\product\10.2.0\Client_2\bin\oraldapclnt10.dll
0x62670000 - 0x6268b000 E:\oracle\product\10.2.0\Client_2\bin\orancrypt10.dll
0x71bb0000 - 0x71bb9000 C:\WINDOWS\system32\WSOCK32.dll
0x62920000 - 0x6296c000 E:\oracle\product\10.2.0\Client_2\bin\oranro10.dll
0x626b0000 - 0x626b7000 E:\oracle\product\10.2.0\Client_2\bin\oranhost10.dll
0x62660000 - 0x62666000 E:\oracle\product\10.2.0\Client_2\bin\orancds10.dll
0x629c0000 - 0x629c8000 E:\oracle\product\10.2.0\Client_2\bin\orantns10.dll
0x60b50000 - 0x60ea8000 E:\oracle\product\10.2.0\Client_2\bin\orapls10.dll
0x11370000 - 0x11379000 E:\oracle\product\10.2.0\Client_2\bin\oraslax10.dll
0x63080000 - 0x63284000 E:\oracle\product\10.2.0\Client_2\bin\oraplp10.dll
0x61ed0000 - 0x61f5b000 E:\oracle\product\10.2.0\Client_2\bin\orahasgen10.dll
0x62ab0000 - 0x62b1a000 E:\oracle\product\10.2.0\Client_2\bin\oraocr10.dll
0x62b20000 - 0x62b60000 E:\oracle\product\10.2.0\Client_2\bin\oraocrb10.dll
0x62980000 - 0x62990000 E:\oracle\product\10.2.0\Client_2\bin\orantcp10.dll
0x11380000 - 0x1141a000 E:\oracle\product\10.2.0\Client_2\bin\orasql10.dll
0x71f60000 - 0x71f64000 C:\WINDOWS\system32\security.dll
0x76df0000 - 0x76e22000 C:\WINDOWS\system32\activeds.dll
0x76dc0000 - 0x76de6000 C:\WINDOWS\system32\adsldpc.dll
0x76b80000 - 0x76bad000 C:\WINDOWS\system32\credui.dll
0x7cd80000 - 0x7d573000 C:\WINDOWS\system32\SHELL32.dll
0x77290000 - 0x772d9000 C:\WINDOWS\system32\SHLWAPI.dll
0x76a80000 - 0x76a98000 C:\WINDOWS\system32\ATL.DLL
0x70ad0000 - 0x70bb6000 C:\WINDOWS\WinSxS\x86_Microsoft.Windows.Common-Controls_6595b64144ccf1df_6.0.100.0_x-ww_8417450B\comctl32.dll
0x76c90000 - 0x76cb4000 C:\WINDOWS\system32\msv1_0.dll
0x11d10000 - 0x11d48000 E:\multiplus\prod7777\muex\skre.dll
0x11d50000 - 0x11d61000 E:\multiplus\prod7777\muex\RELB10.DLL
0x12680000 - 0x129b4000 E:\multiplus\prod7777\muex\LNLB.DLL
0x11d70000 - 0x11e2a000 E:\multiplus\prod7777\muex\skmu.dll
0x11e30000 - 0x11e41000 E:\multiplus\prod7777\muex\MULB10.DLL
0x135c0000 - 0x13632000 E:\multiplus\prod7777\muex\skfi.dll
0x14e50000 - 0x14f98000 E:\multiplus\prod7777\muex\FILB10.DLL
0x157a0000 - 0x1580a000 E:\multiplus\prod7777\muex\skkr.dll
0x15810000 - 0x1582e000 E:\multiplus\prod7777\muex\KRLB10.DLL

VM Arguments:
jvm_args: -Xss1280k -Xmx128M -Xdebug -Xnoagent -Dmulti.environment=E:\multiplus\standard\mas_v1g_3\bin.win32\platform.ini,E:\multiplus\standard\mas_v1g_3\bin.win32\common.ini,E:\multiplus\prod7777\bin.win32\system.ini,E:\multiplus\prod7777\bin.win32\MultiIntegration.ini -Dmulti.global.environment=E:\multiplus\prod7777\bin.win32\MultiSystems.ini -Dmulti.className=com.lindhard.multi.application.MultiApplicationServer
java_command: com.lindhard.multi.launcher.MultiLauncher O_Oko02 OLTP
Launcher Type: SUN_STANDARD

Environment Variables:
JAVA_HOME=C:\Program Files\Java\jdk1.5.0_06
CLASSPATH=C:\IBM\WebSphere MQ\Java\lib\providerutil.jar;C:\IBM\WebSphere MQ\Java\lib\com.ibm.mqjms.jar;C:\IBM\WebSphere MQ\Java\lib\ldap.jar;C:\IBM\WebSphere MQ\Java\lib\jta.jar;C:\IBM\WebSphere MQ\Java\lib\jndi.jar;C:\IBM\WebSphere MQ\Java\lib\jms.jar;C:\IBM\WebSphere MQ\Java\lib\connector.jar;C:\IBM\WebSphere MQ\Java\lib\fscontext.jar;C:\IBM\WebSphere MQ\Java\lib\com.ibm.mq.jar
PATH=E:\Oracle\product\10.2.0\Client_2\bin;C:\Program Files\Legato\nsr\bin;E:\Oracle\product\10.1.0\Client_1\bin;E:\Oracle\product\10.1.0\Client_1\jre\1.4.2\bin\client;E:\Oracle\product\10.1.0\Client_1\jre\1.4.2\bin;C:\IBM\WebSphere MQ\Java\lib;C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\System32\Wbem;C:\IBM\WebSphere MQ\bin;C:\IBM\WebSphere MQ\tools\c\samples\bin;C:\Program Files\Java\jdk1.5.0_06\bin;E:\multiplus\standard\mas_v1g_3\bin.win32;E:\multiplus\standard\mas_v1g_3\muex;E:\multiplus\prod7777\muex;E:\multiplus\prod7777\bin.win32
USERNAME=prod7777
OS=Windows_NT
PROCESSOR_IDENTIFIER=x86 Family 15 Model 2 Stepping 9, GenuineIntel



--------------- S Y S T E M ---------------

OS: Windows Server 2003 family Build 3790

CPU:total 2 family 15, cmov, cx8, fxsr, mmx, sse, sse2, ht

Memory: 4k page, physical 2097151k(2097151k free), swap 4194303k(4194303k free)

vm_info: Java HotSpot(TM) Server VM (1.5.0_06-b05) for windows-x86, built on Nov 10 2005 10:53:00 by "java_re" with MS VC++ 6.0


0:000> !analyze -v
*******************************************************************************
* *
* Exception Analysis *
* *
*******************************************************************************

*** ERROR: Symbol file could not be found. Defaulted to export symbols for net.dll -
*** ERROR: Symbol file could not be found. Defaulted to export symbols for amqxcs2.dll -
*** ERROR: Symbol file could not be found. Defaulted to export symbols for amqzst.dll -

FAULTING_IP:
+0
00000000 ?? ???

EXCEPTION_RECORD: ffffffff -- (.exr ffffffffffffffff)
ExceptionAddress: 00000000
ExceptionCode: 80000003 (Break instruction exception)
ExceptionFlags: 00000000
NumberParameters: 0

FAULTING_THREAD: 00002814

BUGCHECK_STR: 80000003

DEFAULT_BUCKET_ID: APPLICATION_FAULT

PROCESS_NAME: java.exe

ERROR_CODE: (NTSTATUS) 0x80000003 - {UNNTAK} Avbruddspunkt Et avbruddspunkt er n dd.

LAST_CONTROL_TRANSFER: from 77f43741 to 7ffe0304

STACK_TEXT:
0007fc00 77f43741 77e41817 0000270c 00000000 SharedUserData!SystemCallStub+0x4
0007fc04 77e41817 0000270c 00000000 00000000 ntdll!ZwWaitForSingleObject+0xc
0007fc74 77e4168f 0000270c ffffffff 00000000 kernel32!WaitForSingleObjectEx+0xac
0007fc84 6da9c961 0000270c ffffffff 00037dc0 kernel32!WaitForSingleObject+0xf
WARNING: Stack unwind information not available. Following frames may be wrong.
0007fcb8 6db114b8 00000000 ffffffff 0003889c jvm!jmm_GetLastGCStat+0x15bf1
0007fcfc 00401508 00038798 00000000 00000000 jvm!JVM_RegisterPerfMethods+0x4f8b8
0007ff4c 004064dd 00000002 000324a0 00033338 java+0x1508
0007ffc0 77e4f38c 00000000 00000000 7ffdf000 java+0x64dd
0007fff0 00000000 004063fa 00000000 78746341 kernel32!BaseProcessStart+0x23


STACK_COMMAND: ~0s; .ecxr ; kb

FOLLOWUP_IP:
jvm!jmm_GetLastGCStat+15bf1
6da9c961 8b5e18 mov ebx,[esi+0x18]

FAULTING_SOURCE_CODE:


SYMBOL_STACK_INDEX: 4

FOLLOWUP_NAME: MachineOwner

SYMBOL_NAME: jvm!jmm_GetLastGCStat+15bf1

MODULE_NAME: jvm

IMAGE_NAME: jvm.dll

DEBUG_FLR_IMAGE_TIMESTAMP: 4373ba0a

FAILURE_BUCKET_ID: 80000003_jvm!jmm_GetLastGCStat+15bf1

BUCKET_ID: 80000003_jvm!jmm_GetLastGCStat+15bf1

Followup: MachineOwner
---------
0:000> ~*kb

. 0 Id: 2f84.2814 Suspend: 1 Teb: 7ffde000 Unfrozen
ChildEBP RetAddr Args to Child
0007fc00 77f43741 77e41817 0000270c 00000000 SharedUserData!SystemCallStub+0x4
0007fc04 77e41817 0000270c 00000000 00000000 ntdll!ZwWaitForSingleObject+0xc
0007fc74 77e4168f 0000270c ffffffff 00000000 kernel32!WaitForSingleObjectEx+0xac
0007fc84 6da9c961 0000270c ffffffff 00037dc0 kernel32!WaitForSingleObject+0xf
WARNING: Stack unwind information not available. Following frames may be wrong.
0007fcb8 6db114b8 00000000 ffffffff 0003889c jvm!jmm_GetLastGCStat+0x15bf1
0007fcfc 00401508 00038798 00000000 00000000 jvm!JVM_RegisterPerfMethods+0x4f8b8
0007ff4c 004064dd 00000002 000324a0 00033338 java+0x1508
0007ffc0 77e4f38c 00000000 00000000 7ffdf000 java+0x64dd
0007fff0 00000000 004063fa 00000000 78746341 kernel32!BaseProcessStart+0x23

1 Id: 2f84.1c88 Suspend: 1 Teb: 7ffdd000 Unfrozen
ChildEBP RetAddr Args to Child
0ea6fb84 77f43741 77e41817 00002640 00000000 SharedUserData!SystemCallStub+0x4
0ea6fb88 77e41817 00002640 00000000 00000000 ntdll!ZwWaitForSingleObject+0xc
0ea6fbf8 77e4168f 00002640 ffffffff 00000000 kernel32!WaitForSingleObjectEx+0xac
0ea6fc08 6da9c8d7 00002640 ffffffff 007b6d88 kernel32!WaitForSingleObject+0xf
WARNING: Stack unwind information not available. Following frames may be wrong.
0ea6fc3c 6d9418e8 00000001 ffffffff 007b9e88 jvm!jmm_GetLastGCStat+0x15b67
0ea6fc58 6d942095 00000000 00000000 007b6de8 jvm!AsyncGetCallTrace+0x6398
00000000 00000000 00000000 00000000 00000000 jvm!AsyncGetCallTrace+0x6b45

2 Id: 2f84.2898 Suspend: 1 Teb: 7ffdc000 Unfrozen
ChildEBP RetAddr Args to Child
0ec6fc04 77f43741 77e41817 00002640 00000000 SharedUserData!SystemCallStub+0x4
0ec6fc08 77e41817 00002640 00000000 00000000 ntdll!ZwWaitForSingleObject+0xc
0ec6fc78 77e4168f 00002640 ffffffff 00000000 kernel32!WaitForSingleObjectEx+0xac
0ec6fc88 6da9c8d7 00002640 ffffffff 007b6d88 kernel32!WaitForSingleObject+0xf
WARNING: Stack unwind information not available. Following frames may be wrong.
0ec6fcbc 6d9418e8 00000001 ffffffff 007b9e58 jvm!jmm_GetLastGCStat+0x15b67
0ec6fcd8 6d942095 00000001 00000000 007b7838 jvm!AsyncGetCallTrace+0x6398
00000000 00000000 00000000 00000000 00000000 jvm!AsyncGetCallTrace+0x6b45

3 Id: 2f84.1760 Suspend: 1 Teb: 7ffdb000 Unfrozen
ChildEBP RetAddr Args to Child
0ee6fb30 77f43741 77e41817 00002704 00000000 SharedUserData!SystemCallStub+0x4
0ee6fb34 77e41817 00002704 00000000 0ee6fb78 ntdll!ZwWaitForSingleObject+0xc
0ee6fba4 77e4168f 00002704 000003e8 00000000 kernel32!WaitForSingleObjectEx+0xac
0ee6fbb4 6da9c8d7 00002704 000003e8 00037df0 kernel32!WaitForSingleObject+0xf
WARNING: Stack unwind information not available. Following frames may be wrong.
0ee6fbe8 6db399c3 00000001 000003e8 00037ec0 jvm!jmm_GetLastGCStat+0x15b67
0ee6fc2c 6daab055 00002608 00000002 6db39792 jvm!JVM_RegisterUnsafeMethods+0x12203
0ee6ff84 77bc91ed 0084cef0 00000000 00000000 jvm!jmm_GetLastGCStat+0x242e5
0ee6ffb8 77e4a990 007b7758 00000000 00000000 msvcrt!_endthreadex+0x95
0ee6ffec 00000000 77bc917e 007b7758 00000000 kernel32!BaseThreadStart+0x34

4 Id: 2f84.c38 Suspend: 1 Teb: 7ffda000 Unfrozen
ChildEBP RetAddr Args to Child
0f06f7cc 77f43741 77e41817 000025f4 00000000 SharedUserData!SystemCallStub+0x4
0f06f7d0 77e41817 000025f4 00000000 00000000 ntdll!ZwWaitForSingleObject+0xc
0f06f840 77e4168f 000025f4 ffffffff 00000000 kernel32!WaitForSingleObjectEx+0xac
0f06f850 6daa49de 000025f4 ffffffff 0084f090 kernel32!WaitForSingleObject+0xf
WARNING: Stack unwind information not available. Following frames may be wrong.
00000000 00000000 00000000 00000000 00000000 jvm!jmm_GetLastGCStat+0x1dc6e

5 Id: 2f84.2594 Suspend: 1 Teb: 7ffd9000 Unfrozen
ChildEBP RetAddr Args to Child
0f46fb1c 77f43741 77e41817 0000256c 00000000 SharedUserData!SystemCallStub+0x4
0f46fb20 77e41817 0000256c 00000000 00000000 ntdll!ZwWaitForSingleObject+0xc
0f46fb90 77e4168f 0000256c ffffffff 00000000 kernel32!WaitForSingleObjectEx+0xac
0f46fba0 6daa49de 0000256c ffffffff 0084fd30 kernel32!WaitForSingleObject+0xf
WARNING: Stack unwind information not available. Following frames may be wrong.
00000000 00000000 00000000 00000000 00000000 jvm!jmm_GetLastGCStat+0x1dc6e

6 Id: 2f84.17e4 Suspend: 1 Teb: 7ffd8000 Unfrozen
ChildEBP RetAddr Args to Child
0f66fce8 77f43741 77e41817 000024cc 00000000 SharedUserData!SystemCallStub+0x4
0f66fcec 77e41817 000024cc 00000000 00000000 ntdll!ZwWaitForSingleObject+0xc
0f66fd5c 77e4168f 000024cc ffffffff 00000000 kernel32!WaitForSingleObjectEx+0xac
0f66fd6c 6daa9c73 000024cc ffffffff 0f078860 kernel32!WaitForSingleObject+0xf
WARNING: Stack unwind information not available. Following frames may be wrong.
0f66fd98 6daa9d67 00000001 6daa757c 00000000 jvm!jmm_GetLastGCStat+0x22f03
0f66fda0 6daa757c 00000000 0f078860 0f66ff84 jvm!jmm_GetLastGCStat+0x22ff7
0f66fe58 77e4f1c5 0f66ff74 77f45a34 77f89570 jvm!jmm_GetLastGCStat+0x2080c
0f66fe70 77bbbb26 00030000 00000000 00000098 kernel32!VirtualProtectEx+0x1e
0f66fe90 6daa7956 00000098 00000098 6db0e5e2 msvcrt!malloc+0x57
0f66fe9c 6db0e5e2 0f078860 0f078860 0f078860 jvm!jmm_GetLastGCStat+0x20be6
0f66fec4 6db0e5ab 0f078860 6daa8b82 00000000 jvm!JVM_RegisterPerfMethods+0x4c9e2
0f66fecc 6daa8b82 00000000 00000000 00000000 jvm!JVM_RegisterPerfMethods+0x4c9ab
0f66ff38 77f467f9 77bbb9be 00030000 00000000 jvm!jmm_GetLastGCStat+0x21e12
0f66fff4 007b7758 00000000 00000000 00000000 ntdll!RtlFreeHeap+0x642
77bc917e 00000000 ffffd2a2 baec35ff 15ff77be 0x7b7758

7 Id: 2f84.22ec Suspend: 1 Teb: 7ffd7000 Unfrozen
ChildEBP RetAddr Args to Child
0f86fce8 77f43741 77e41817 000026a4 00000000 SharedUserData!SystemCallStub+0x4
0f86fcec 77e41817 000026a4 00000000 00000000 ntdll!ZwWaitForSingleObject+0xc
0f86fd5c 77e4168f 000026a4 ffffffff 00000000 kernel32!WaitForSingleObjectEx+0xac
0f86fd6c 6da9c961 000026a4 ffffffff 00032540 kernel32!WaitForSingleObject+0xf
WARNING: Stack unwind information not available. Following frames may be wrong.
0f86fda0 6d8fd922 00000000 ffffffff 00032540 jvm!jmm_GetLastGCStat+0x15bf1
0f86fdb8 6d8fedd5 00000000 0f0795a0 0f86ff84 jvm+0x7d922
00000000 00000000 00000000 00000000 00000000 jvm+0x7edd5

8 Id: 2f84.214c Suspend: 1 Teb: 7ffd6000 Unfrozen
ChildEBP RetAddr Args to Child
0fa6fc68 77f43741 77e41817 0000269c 00000000 SharedUserData!SystemCallStub+0x4
0fa6fc6c 77e41817 0000269c 00000000 00000000 ntdll!ZwWaitForSingleObject+0xc
0fa6fcdc 77e4168f 0000269c ffffffff 00000000 kernel32!WaitForSingleObjectEx+0xac
0fa6fcec 6da9c961 0000269c ffffffff 00032540 kernel32!WaitForSingleObject+0xf
WARNING: Stack unwind information not available. Following frames may be wrong.
0fa6fd20 6d8fd922 00000000 ffffffff 00032540 jvm!jmm_GetLastGCStat+0x15bf1
0fa6fd38 6d8fedd5 00000000 0f07a450 0fa6ff84 jvm+0x7d922
00000000 00000000 00000000 00000000 00000000 jvm+0x7edd5

9 Id: 2f84.db8 Suspend: 1 Teb: 7ffd5000 Unfrozen
ChildEBP RetAddr Args to Child
0fc6fbe8 77f43741 77e41817 0000269c 00000000 SharedUserData!SystemCallStub+0x4
0fc6fbec 77e41817 0000269c 00000000 00000000 ntdll!ZwWaitForSingleObject+0xc
0fc6fc5c 77e4168f 0000269c ffffffff 00000000 kernel32!WaitForSingleObjectEx+0xac
0fc6fc6c 6da9c961 0000269c ffffffff 00032540 kernel32!WaitForSingleObject+0xf
WARNING: Stack unwind information not available. Following frames may be wrong.
0fc6fca0 6d8fd922 00000000 ffffffff 00032540 jvm!jmm_GetLastGCStat+0x15bf1
0fc6fcb8 6d8fedd5 00000000 0f07b2c8 0fc6ff84 jvm+0x7d922
00000000 00000000 00000000 00000000 00000000 jvm+0x7edd5

10 Id: 2f84.1a38 Suspend: 1 Teb: 7ffd4000 Unfrozen
ChildEBP RetAddr Args to Child
0fe6fba0 77f43741 77e41817 00002778 00000000 SharedUserData!SystemCallStub+0x4
0fe6fba4 77e41817 00002778 00000000 00000000 ntdll!ZwWaitForSingleObject+0xc
0fe6fc14 77e4168f 00002778 ffffffff 00000000 kernel32!WaitForSingleObjectEx+0xac
0fe6fc24 6da9c8d7 00002778 ffffffff 0f07c5d0 kernel32!WaitForSingleObject+0xf
WARNING: Stack unwind information not available. Following frames may be wrong.
0fe6fc58 6da802c5 00000001 ffffffff 00000000 jvm!jmm_GetLastGCStat+0x15b67
0fe6ff84 77bc91ed 0f07c5d0 00000000 00000000 jvm!JVM_FindSignal+0xb42a5
0fe6ffb8 77e4a990 007b7758 00000000 00000000 msvcrt!_endthreadex+0x95
0fe6ffec 00000000 77bc917e 007b7758 00000000 kernel32!BaseThreadStart+0x34

11 Id: 2f84.aac Suspend: 1 Teb: 7ffaf000 Unfrozen
ChildEBP RetAddr Args to Child
1006fb64 77f4262b 77e418ea 00000000 1006fba8 SharedUserData!SystemCallStub+0x4
1006fb68 77e418ea 00000000 1006fba8 00000000 ntdll!NtDelayExecution+0xc
1006fbd0 77e416ee 00001388 00000000 6daaafe7 kernel32!SleepEx+0x68
1006fbdc 6daaafe7 00001388 00000032 00000000 kernel32!Sleep+0xb
WARNING: Stack unwind information not available. Following frames may be wrong.
1006fbf8 6db0df7f 0f07d988 00001388 0f07e2b8 jvm!jmm_GetLastGCStat+0x24277
00000000 00000000 00000000 00000000 00000000 jvm!JVM_RegisterPerfMethods+0x4c37f

12 Id: 2f84.2df0 Suspend: 1 Teb: 7ffae000 Unfrozen
ChildEBP RetAddr Args to Child
1153ecf0 77f43741 71b23ac3 00002190 00000001 SharedUserData!SystemCallStub+0x4
1153ecf4 71b23ac3 00002190 00000001 1153ed1c ntdll!ZwWaitForSingleObject+0xc
1153ed30 71b2b13e 00002190 000021bc 00000000 mswsock!SockWaitForSingleObject+0x19b
1153eda8 71c0fdf1 000021bc 1153ede0 00000001 mswsock!WSPRecv+0x1f0
1153edf0 6d5369fb 000021bc 10233ec8 00002000 ws2_32!recv+0x81
WARNING: Stack unwind information not available. Following frames may be wrong.
1153f618 0085a482 10233ec8 1153f67c 1153f678 net!Java_java_net_SocketInputStream_socketRead0+0x140
1153f658 00852a3f 00000000 008573d9 00000000 0x85a482
1153f69c 00852a3f 00000000 00000000 00000000 0x852a3f
1153f6e8 00852a3f 00000000 00002000 00000000 0x852a3f
1153f720 00852a3f 00002000 00000000 068df640 0x852a3f
1153f754 00852a3f 00000000 00000000 00000000 0x852a3f
1153f794 00852a3f 00000000 068c65f0 00000000 0x852a3f
1153f7e4 00852a3f 00000000 00000000 00000000 0x852a3f
1153f830 00852a3f 00002000 00000000 068c7ea8 0x852a3f
1153f864 00888930 00000000 00000000 00000000 0x852a3f
00000000 00000000 00000000 00000000 00000000 0x888930

13 Id: 2f84.2d98 Suspend: 1 Teb: 7ffad000 Unfrozen
ChildEBP RetAddr Args to Child
11e9f3ec 77f43741 77e41817 000021e4 00000000 SharedUserData!SystemCallStub+0x4
11e9f3f0 77e41817 000021e4 00000000 11e9f434 ntdll!ZwWaitForSingleObject+0xc
11e9f460 77e4168f 000021e4 00002710 00000000 kernel32!WaitForSingleObjectEx+0xac
11e9f470 4e866955 000021e4 00002710 10c862a4 kernel32!WaitForSingleObject+0xf
WARNING: Stack unwind information not available. Following frames may be wrong.
11e9f4a8 77e4183c 77e4168f 00002200 00000000 amqxcs2!xcsWaitEventSem+0x195
11e9f538 4f0f5996 10c862a4 4f0f4c80 10c862a4 kernel32!WaitForSingleObjectEx+0xdc
11e9f564 4f0f1c20 0f1f2208 11e9f594 11e9fbbc amqzst!ziiVSendReceiveAgent+0xc46
11e9f5c0 4f100366 0f1f2208 0000088a 11e9f988 amqzst+0x1c20
00000000 00000000 00000000 00000000 00000000 amqzst!rzstMQGET+0x846

14 Id: 2f84.2fa0 Suspend: 1 Teb: 7ffac000 Unfrozen
ChildEBP RetAddr Args to Child
1209e86c 77f43741 77e41817 00001eec 00000000 SharedUserData!SystemCallStub+0x4
1209e870 77e41817 00001eec 00000000 00000000 ntdll!ZwWaitForSingleObject+0xc
1209e8e0 77e4168f 00001eec ffffffff 00000000 kernel32!WaitForSingleObjectEx+0xac
1209e8f0 6db33cfa 00001eec ffffffff 6dbe1014 kernel32!WaitForSingleObject+0xf
WARNING: Stack unwind information not available. Following frames may be wrong.
1209e9ac 6daaa25f 00000000 00000000 1209f77c jvm!JVM_RegisterUnsafeMethods+0xc53a
1209ea1c 6daabd22 00000001 77bc4db1 1209ea4c jvm!jmm_GetLastGCStat+0x234ef
00000000 00000000 00000000 00000000 00000000 jvm!jmm_GetLastGCStat+0x24fb2