Java APIs

We seem to be getting a SIGSEGV in the native code of a Compiler Thread (typically named CompilerThread1).

My problem is most similar to.

http://forum.java.sun.com/thread.jspa?forumID=37&threadID=606432

Except we are not running a 64 bit JVM. We are running JVM 1.5.0_06-b05 on a Red Hat Enterprise Linux environment.

We started seeing this problem after we set -Xms500m -Xmx2000m. Before we implemented these settings, we were seeing OutOfMemoryExceptions while the applicaiton server was running. On the production environment, this error comes up when we are under heavy load. We have had a difficult time reproducing this error in a non-production environment, but we will continue to try to reproduce it on a consistent basis.

This PDF seems to suggest that I am seeing a bug with the compiler.

http://java.sun.com/j2se/1.5/pdf/jdk50_ts_guide.pdf

For the next step we will disable compilation for the method named by the current CompileTask. We will also try switching to a client hotspot VM. However, because I have had trouble consistently reproducing the bug, I would like to get feedback from everyone. Is this really a bug with the compiler? Could it be a problem with memory leak or somekind of resource leak? Could it be a problem with Lucene specifcally?

#
# An unexpected error has been detected by HotSpot Virtual Machine:
#
# SIGSEGV (0xb) at pc=0xb6e97cd1, pid=27543, tid=599825328
#
# Java VM: Java HotSpot(TM) Server VM (1.5.0_06-b05 mixed mode)
# Problematic frame:
# V [libjvm.so+0x3c9cd1]
#

--------------- T H R E A D ---------------

Current thread (0x080f3638): JavaThread "CompilerThread1" daemon [_thread_in_native, id=27550]

siginfo:si_signo=11, si_errno=0, si_code=1, si_addr=0x00000000

Registers:
EAX=0x00000000, EBX=0xb70797f0, ECX=0x23c08610, EDX=0x21807fc4
ESP=0x23c08584, EBP=0x23c085a8, ESI=0x0856c918, EDI=0x23c08610
EIP=0xb6e97cd1, CR2=0x00000000, EFLAGS=0x00010216

Top of Stack: (sp=0x23c08584)
0x23c08584: 23c08bc0 23c08bd0 b6e97c16 b70797f0
0x23c08594: 00000000 23c085c8 b70797f0 0856c918
0x23c085a4: 0856c8d4 23c085d8 b6e97d75 0856d01c
0x23c085b4: 21807fc4 0856d01c 23c08610 0856d080
0x23c085c4: 21801884 0856c8d4 b70797f0 0856d01c
0x23c085d4: 00000000 23c08638 b6e98258 0856d01c
0x23c085e4: 0856c918 0856d01c 23c08610 00000023
0x23c085f4: 218010ac 23c08638 b6f1cbf1 21801884

Instructions: (pc=0xb6e97cd1)
0xb6e97cc1: 10 eb de 31 c0 eb 8e 8b 42 04 83 ec 0c 8b 40 08
0xb6e97cd1: 8b 08 50 ff 51 28 83 c4 10 e9 48 ff ff ff 90 55

Stack: [0x23b89000,0x23c0a000), sp=0x23c08584, free space=509k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
V [libjvm.so+0x3c9cd1]
V [libjvm.so+0x3c9d75]
V [libjvm.so+0x3ca258]
V [libjvm.so+0x4511fd]
V [libjvm.so+0x450e90]
V [libjvm.so+0x1a11c0]
V [libjvm.so+0x19e432]
V [libjvm.so+0x1474b3]
V [libjvm.so+0x1a6929]
V [libjvm.so+0x1a6281]
V [libjvm.so+0x4c8366]
V [libjvm.so+0x4c2ba3]
V [libjvm.so+0x424338]
C [libpthread.so.0+0x4dd8]


Current CompileTask:
opto:972 org.apache.lucene.index.IndexReader$1.doBody()Ljava/lang/Object; (99 bytes)


--------------- P R O C E S S ---------------

Java Threads: ( => current thread )
0x09b08440 JavaThread "PingSender" daemon [_thread_blocked, id=27683]
0x097fa108 JavaThread "PingWaiter" daemon [_thread_blocked, id=27682]
0x08cfaae8 JavaThread "TimeScheduler.Thread" daemon [_thread_blocked, id=27678]
0x09b784d0 JavaThread "TimeScheduler.Thread" daemon [_thread_blocked, id=27677]
0x09cdfaa0 JavaThread "MERGE2.FindSubgroups thread" daemon [_thread_blocked, id=27671]
0x097fa2c8 JavaThread "Timer-3" daemon [_thread_blocked, id=27647]
0x097f9c90 JavaThread "http-0.0.0.0-8080-2" daemon [_thread_in_native, id=27646]
0x08116da0 JavaThread "http-0.0.0.0-8080-1" daemon [_thread_blocked, id=27644]
0x09b8aee8 JavaThread "TP-Monitor" daemon [_thread_blocked, id=27643]
0x09b8aa88 JavaThread "TP-Processor4" daemon [_thread_in_native, id=27642]
0x09c35eb0 JavaThread "TP-Processor3" daemon [_thread_blocked, id=27641]
0x09b2e8e0 JavaThread "TP-Processor2" daemon [_thread_blocked, id=27640]
0x09b9a760 JavaThread "TP-Processor1" daemon [_thread_blocked, id=27639]
0x09b2ec60 JavaThread "http-0.0.0.0-8080" daemon [_thread_in_native, id=27638]
0x09c30610 JavaThread "JBossLifeThread" [_thread_blocked, id=27637]
0x082a8990 JavaThread "UDP mcast receiver" daemon [_thread_in_native, id=27633]
0x082a8180 JavaThread "UDP.UcastReceiverThread" daemon [_thread_in_native, id=27632]
0x080ca0a0 JavaThread "TimeScheduler.Thread" daemon [_thread_blocked, id=27631]
0x09d42dc0 JavaThread "Thread-48" daemon [_thread_blocked, id=27630]
0x080c9ba8 JavaThread "DistributedIndexRetryTimer" daemon [_thread_blocked, id=27629]
0x09fbbef8 JavaThread "DistributedIndexDispatchThread" [_thread_blocked, id=27628]
0x09d42fa0 JavaThread "Timer-2" daemon [_thread_blocked, id=27627]
0x09b8f160 JavaThread "LuceneSearchServiceIndexQueueThread" [_thread_in_vm, id=27626]
0x09d43ae0 JavaThread "DefaultQuartzScheduler_QuartzSchedulerThread" [_thread_blocked, id=27624]
0x08a8d468 JavaThread "DefaultQuartzScheduler_Worker-9" [_thread_blocked, id=27623]
0x09914778 JavaThread "DefaultQuartzScheduler_Worker-8" [_thread_blocked, id=27622]
0x089c8e20 JavaThread "DefaultQuartzScheduler_Worker-7" [_thread_blocked, id=27621]
0x089c7f00 JavaThread "DefaultQuartzScheduler_Worker-6" [_thread_blocked, id=27620]
0x08ae0310 JavaThread "DefaultQuartzScheduler_Worker-5" [_thread_blocked, id=27619]
0x086747a8 JavaThread "DefaultQuartzScheduler_Worker-4" [_thread_blocked, id=27618]
0x089869f8 JavaThread "DefaultQuartzScheduler_Worker-3" [_thread_blocked, id=27617]
0x081fa918 JavaThread "DefaultQuartzScheduler_Worker-2" [_thread_blocked, id=27616]
0x08c31048 JavaThread "DefaultQuartzScheduler_Worker-1" [_thread_blocked, id=27615]
0x09b27f98 JavaThread "DefaultQuartzScheduler_Worker-0" [_thread_blocked, id=27614]
0x097d7aa8 JavaThread "UDP mcast receiver" daemon [_thread_in_native, id=27609]
0x08c98710 JavaThread "UDP.UcastReceiverThread" daemon [_thread_in_native, id=27608]
0x08c98a10 JavaThread "TimeScheduler.Thread" daemon [_thread_blocked, id=27607]
0x097d77e8 JavaThread "Thread-39" daemon [_thread_blocked, id=27606]
0x097da8e8 JavaThread "UpHandler (GMS)" daemon [_thread_blocked, id=27605]
0x09745420 JavaThread "DownHandler (GMS)" daemon [_thread_blocked, id=27604]
0x08c947a0 JavaThread "UpHandler (FRAG)" daemon [_thread_blocked, id=27603]
0x08c94218 JavaThread "DownHandler (FRAG)" daemon [_thread_blocked, id=27602]
0x08d1c3d0 JavaThread "UpHandler (STABLE)" daemon [_thread_blocked, id=27601]
0x089b36d0 JavaThread "DownHandler (STABLE)" daemon [_thread_blocked, id=27600]
0x0820ff00 JavaThread "UpHandler (UNICAST)" daemon [_thread_blocked, id=27599]
0x0820f978 JavaThread "DownHandler (UNICAST)" daemon [_thread_blocked, id=27598]
0x0820f418 JavaThread "UpHandler (NAKACK)" daemon [_thread_blocked, id=27597]
0x09d06898 JavaThread "DownHandler (NAKACK)" daemon [_thread_blocked, id=27596]
0x082e63d0 JavaThread "UpHandler (VERIFY_SUSPECT)" daemon [_thread_blocked, id=27595]
0x09bf7908 JavaThread "DownHandler (VERIFY_SUSPECT)" daemon [_thread_blocked, id=27594]
0x08d258c0 JavaThread "UpHandler (FD)" daemon [_thread_blocked, id=27593]
0x09c9cc78 JavaThread "DownHandler (FD)" daemon [_thread_blocked, id=27592]
0x09c9c6f8 JavaThread "UpHandler (PING)" daemon [_thread_blocked, id=27591]
0x08c9a400 JavaThread "DownHandler (PING)" daemon [_thread_blocked, id=27590]
0x09bf8060 JavaThread "DownHandler (UDP)" daemon [_thread_blocked, id=27589]
0x09c49bd8 JavaThread "JCA PoolFiller" [_thread_blocked, id=27588]
0x09e22048 JavaThread "IdleRemover" [_thread_blocked, id=27587]
0x082ae378 JavaThread "UpHandler (GMS)" daemon [_thread_blocked, id=27586]
0x082addf0 JavaThread "DownHandler (GMS)" daemon [_thread_blocked, id=27585]
0x085b67e8 JavaThread "UpHandler (FRAG)" daemon [_thread_blocked, id=27584]
0x08237fc8 JavaThread "DownHandler (FRAG)" daemon [_thread_blocked, id=27583]
0x09b6deb0 JavaThread "UpHandler (STABLE)" daemon [_thread_blocked, id=27582]
0x09b9f7c8 JavaThread "DownHandler (STABLE)" daemon [_thread_blocked, id=27581]
0x0986be10 JavaThread "UpHandler (UNICAST)" daemon [_thread_blocked, id=27580]
0x09c17a40 JavaThread "DownHandler (UNICAST)" daemon [_thread_blocked, id=27579]
0x089d5d68 JavaThread "UpHandler (NAKACK)" daemon [_thread_blocked, id=27578]
0x08c71320 JavaThread "DownHandler (NAKACK)" daemon [_thread_blocked, id=27577]
0x080dfb28 JavaThread "UpHandler (VERIFY_SUSPECT)" daemon [_thread_blocked, id=27576]
0x0987a9e8 JavaThread "DownHandler (VERIFY_SUSPECT)" daemon [_thread_blocked, id=27575]
0x09cf61c8 JavaThread "UpHandler (FD)" daemon [_thread_blocked, id=27574]
0x08568f50 JavaThread "DownHandler (FD)" daemon [_thread_blocked, id=27573]
0x08568ae8 JavaThread "UpHandler (PING)" daemon [_thread_blocked, id=27572]
0x08c3c840 JavaThread "DownHandler (PING)" daemon [_thread_blocked, id=27571]
0x09748998 JavaThread "DownHandler (UDP)" daemon [_thread_blocked, id=27570]
0x09e965d8 JavaThread "WorkManager(3)-1" daemon [_thread_blocked, id=27569]
0x08b33828 JavaThread "ContainerBackgroundProcessor[StandardEngine[jboss.web]]" daemon [_thread_blocked, id=27567]
0x08ced500 JavaThread "PooledInvokerAcceptor#0-4445" [_thread_in_native, id=27566]
0x08b3c6f8 JavaThread "RMI TCP Accept-4444" daemon [_thread_in_native, id=27565]
0x08beec00 JavaThread "ClassLoadingPool(2)-1" daemon [_thread_in_native, id=27564]
0x08cf2b28 JavaThread "JBoss System Threads(1)-1" daemon [_thread_in_native, id=27563]
0x08b3ebb8 JavaThread "GC Daemon" daemon [_thread_blocked, id=27562]
0x08c76f48 JavaThread "RMI Reaper" [_thread_blocked, id=27561]
0x08c77570 JavaThread "Timer-1" daemon [_thread_blocked, id=27560]
0x08c77968 JavaThread "RMI TCP Accept-1098" daemon [_thread_in_native, id=27559]
0x08c42148 JavaThread "ScannerThread" daemon [_thread_blocked, id=27558]
0x08c861a0 JavaThread "Dispatcher-Thread-2" daemon [_thread_blocked, id=27557]
0x08bc47f8 JavaThread "Timer-0" daemon [_thread_blocked, id=27555]
0x0805c940 JavaThread "DestroyJavaVM" [_thread_blocked, id=27543]
0x080f4ac0 JavaThread "Low Memory Detector" daemon [_thread_blocked, id=27551]
=>0x080f3638 JavaThread "CompilerThread1" daemon [_thread_in_native, id=27550]
0x080f25d8 JavaThread "CompilerThread0" daemon [_thread_blocked, id=27549]
0x080f1330 JavaThread "AdapterThread" daemon [_thread_blocked, id=27548]
0x080f0478 JavaThread "Signal Dispatcher" daemon [_thread_blocked, id=27547]
0x080e5c68 JavaThread "Finalizer" daemon [_thread_blocked, id=27546]
0x080e57d8 JavaThread "Reference Handler" daemon [_thread_blocked, id=27545]

Other Threads:
0x080e3348 VMThread [id=27544]
0x080f5f60 WatcherThread [id=27552]

VM state:not at safepoint (normal execution)

VM Mutex/Monitor currently owned by a thread: None

Heap
def new generation total 51328K, used 7185K [0x24a80000, 0x28230000, 0x328b0000)
eden space 45632K, 3% used [0x24a80000, 0x24bf4588, 0x27710000)
from space 5696K, 100% used [0x27ca0000, 0x28230000, 0x28230000)
to space 5696K, 0% used [0x27710000, 0x27710000, 0x27ca0000)
tenured generation total 455168K, used 44152K [0x328b0000, 0x4e530000, 0xa1a80000)
the space 455168K, 9% used [0x328b0000, 0x353ce2a0, 0x353ce400, 0x4e530000)
compacting perm gen total 36608K, used 36539K [0xa1a80000, 0xa3e40000, 0xb1a80000)
the space 36608K, 99% used [0xa1a80000, 0xa3e2ef78, 0xa3e2f000, 0xa3e40000)
No shared spaces configured.

Dynamic libraries:
08048000-08057000 r-xp 00000000 08:03 48004 /opt/jdk1.5.0_06/bin/java
08057000-08059000 rwxp 0000e000 08:03 48004 /opt/jdk1.5.0_06/bin/java
08059000-0a0db000 rwxp 00000000 00:00 0
19cf6000-19cf9000 ---p 00000000 00:00 0
19cf9000-19d77000 rwxp 00003000 00:00 0
19d77000-19d7a000 ---p 00000000 00:00 0
19d7a000-19df8000 rwxp 00003000 00:00 0
19df8000-19dfb000 ---p 00000000 00:00 0
19dfb000-19e79000 rwxp 00003000 00:00 0
19e79000-19e7c000 ---p 00081000 00:00 0
19e7c000-19efa000 rwxp 00084000 00:00 0
19efa000-19efd000 ---p 00000000 00:00 0
19efd000-19f7b000 rwxp 00003000 00:00 0
19f7b000-19f7e000 ---p 00000000 00:00 0
19f7e000-19ffc000 rwxp 00003000 00:00 0
19ffc000-19fff000 ---p 00081000 00:00 0
19fff000-1a07d000 rwxp 00084000 00:00 0
1a07d000-1a080000 ---p 00000000 00:00 0
1a080000-1a0fe000 rwxp 00003000 00:00 0
1a0fe000-1a101000 ---p 00081000 00:00 0
1a101000-1a17f000 rwxp 00084000 00:00 0
1a17f000-1a182000 ---p 00000000 00:00 0
1a182000-1a200000 rwxp 00003000 00:00 0
1a27d000-1a280000 ---p 00000000 00:00 0
1a280000-1a2fe000 rwxp 00003000 00:00 0
1a2fe000-1a301000 ---p 00081000 00:00 0
1a301000-1a37f000 rwxp 00084000 00:00 0
1a37f000-1a382000 ---p 00000000 00:00 0
1a382000-1a400000 rwxp 00003000 00:00 0
1a47b000-1a47e000 ---p 00000000 00:00 0
1a47e000-1a4fc000 rwxp 00003000 00:00 0
1a4fc000-1a4ff000 ---p 00081000 00:00 0
1a4ff000-1a57d000 rwxp 00084000 00:00 0
1a57d000-1a580000 ---p 00000000 00:00 0
1a580000-1a5fe000 rwxp 00003000 00:00 0
1a5fe000-1a601000 ---p 00082000 00:00 0
1a601000-1a67f000 rwxp 00085000 00:00 0
1a67f000-1a682000 ---p 00103000 00:00 0
1a682000-1a721000 rwxp 00106000 00:00 0
1a721000-1a800000 ---p 00000000 00:00 0
1a87f000-1a882000 ---p 00000000 00:00 0
1a882000-1a9f9000 rwxp 00003000 00:00 0
1a9f9000-1aa00000 ---p 0011c000 00:00 0
1aa2c000-1aa2f000 ---p 00000000 00:00 0
1aa2f000-1aaad000 rwxp 00003000 00:00 0
1aaad000-1aab0000 ---p 00081000 00:00 0
1aab0000-1ab2e000 rwxp 00084000 00:00 0
1ab2e000-1ab31000 ---p 00000000 00:00 0
1ab31000-1abaf000 rwxp 00003000 00:00 0
1abaf000-1abb2000 ---p 00081000 00:00 0
1abb2000-1ac30000 rwxp 00084000 00:00 0
1ac30000-1ac33000 ---p 00102000 00:00 0
1ac33000-1acb1000 rwxp 00105000 00:00 0
1acba000-1acbd000 ---p 00000000 00:00 0
1acbd000-1ad3b000 rwxp 00003000 00:00 0
1ad7f000-1ad82000 ---p 00000000 00:00 0
1ad82000-1af00000 rwxp 00003000 00:00 0
1af7d000-1af80000 ---p 00042000 00:00 0
1af80000-1affe000 rwxp 00045000 00:00 0
1affe000-1b001000 ---p 00081000 00:00 0
1b001000-1b07f000 rwxp 00084000 00:00 0
1b07f000-1b082000 ---p 00102000 00:00 0
1b082000-1b200000 rwxp 00105000 00:00 0
1b27f000-1b282000 ---p 00000000 00:00 0
1b282000-1b400000 rwxp 00003000 00:00 0
1b452000-1b455000 ---p 00000000 00:00 0
1b455000-1b4d3000 rwxp 00003000 00:00 0
1b4d3000-1b4d6000 ---p 00081000 00:00 0
1b4d6000-1b554000 rwxp 00084000 00:00 0
1b554000-1b557000 ---p 00081000 00:00 0
1b557000-1b5d5000 rwxp 00084000 00:00 0
1b5d5000-1b5d8000 ---p 00000000 00:00 0
1b5d8000-1b656000 rwxp 00003000 00:00 0
1b656000-1b659000 ---p 00081000 00:00 0
1b659000-1b6d7000 rwxp 00084000 00:00 0
1b6d7000-1b6da000 ---p 00000000 00:00 0
1b6da000-1b758000 rwxp 00003000 00:00 0
1b758000-1b75b000 ---p 00081000 00:00 0
1b75b000-1b7d9000 rwxp 00084000 00:00 0
1b7d9000-1b7dc000 ---p 00000000 00:00 0
1b7dc000-1b85a000 rwxp 00003000 00:00 0
1b85a000-1b85d000 ---p 00081000 00:00 0
1b85d000-1b8db000 rwxp 00084000 00:00 0
1b8db000-1b8de000 ---p 00102000 00:00 0
1b8de000-1b95c000 rwxp 00105000 00:00 0
1b95c000-1b95f000 ---p 00081000 00:00 0
1b95f000-1b9dd000 rwxp 00084000 00:00 0
1b9dd000-1b9e0000 ---p 00102000 00:00 0
1b9e0000-1ba5e000 rwxp 00105000 00:00 0
1ba5e000-1ba61000 ---p 00000000 00:00 0
1ba61000-1badf000 rwxp 00003000 00:00 0
1badf000-1bae2000 ---p 00081000 00:00 0
1bae2000-1bb60000 rwxp 00084000 00:00 0
1bb60000-1bb63000 ---p 00102000 00:00 0
1bb63000-1bbe1000 rwxp 00105000 00:00 0
1bbe1000-1bbe4000 ---p 00183000 00:00 0
1bbe4000-1bc62000 rwxp 00186000 00:00 0
1bc62000-1bc65000 ---p 00204000 00:00 0
1bc65000-1bce3000 rwxp 00207000 00:00 0
1bce3000-1bce6000 ---p 00081000 00:00 0
1bce6000-1bd64000 rwxp 00084000 00:00 0
1bd64000-1bd67000 ---p 00102000 00:00 0
1bd67000-1bde5000 rwxp 00105000 00:00 0
1bde5000-1bde8000 ---p 00183000 00:00 0
1bde8000-1be66000 rwxp 00186000 00:00 0
1be66000-1be69000 ---p 00204000 00:00 0
1be69000-1bee7000 rwxp 00207000 00:00 0
1bee7000-1beea000 ---p 00081000 00:00 0
1beea000-1bf68000 rwxp 00084000 00:00 0
1bf68000-1bf6b000 ---p 00102000 00:00 0
1bf6b000-1bfe9000 rwxp 00105000 00:00 0
1bfe9000-1bfec000 ---p 00183000 00:00 0
1bfec000-1c06a000 rwxp 00186000 00:00 0
1c06a000-1c06d000 ---p 00000000 00:00 0
1c06d000-1c0eb000 rwxp 00003000 00:00 0
1c0eb000-1c0ee000 ---p 00081000 00:00 0
1c0ee000-1c16c000 rwxp 00084000 00:00 0
1c16c000-1c16f000 ---p 00102000 00:00 0
1c16f000-1c1ed000 rwxp 00105000 00:00 0
1c1ed000-1c1f0000 ---p 00183000 00:00 0
1c1f0000-1c26e000 rwxp 00186000 00:00 0
1c27f000-1c282000 ---p 00000000 00:00 0
1c282000-1c400000 rwxp 00003000 00:00 0
1c460000-1c463000 ---p 00042000 00:00 0
1c463000-1c4e1000 rwxp 00045000 00:00 0
1c4e1000-1c4e4000 ---p 00081000 00:00 0
1c4e4000-1c562000 rwxp 00084000 00:00 0
1c562000-1c565000 ---p 00102000 00:00 0
1c565000-1c5e3000 rwxp 00105000 00:00 0
1c5e3000-1c5e6000 ---p 00183000 00:00 0
1c5e6000-1c664000 rwxp 00186000 00:00 0
1c665000-1c668000 ---p 00000000 00:00 0
1c668000-1c6e6000 rwxp 00003000 00:00 0
1c6e6000-1c6e9000 ---p 00081000 00:00 0
1c6e9000-1c767000 rwxp 00084000 00:00 0
1c77f000-1c782000 ---p 00000000 00:00 0
1c782000-1c900000 rwxp 00003000 00:00 0
1c97f000-1c982000 ---p 00000000 00:00 0
1c982000-1cafd000 rwxp 00003000 00:00 0
1cafd000-1cb00000 ---p 00029000 00:00 0
1cb00000-1cbff000 rwxp 00000000 00:00 0
1cbff000-1cc00000 ---p 00000000 00:00 0
1cc7f000-1cc82000 ---p 00000000 00:00 0
1cc82000-1cdf7000 rwxp 00003000 00:00 0
1cdf7000-1ce00000 ---p 000f7000 00:00 0
1ce2e000-1ce31000 ---p 00000000 00:00 0
1ce31000-1ceaf000 rwxp 00003000 00:00 0
1ceaf000-1cece000 r-xs 00000000 08:02 1966130 /var/lms/lms-server/tmp/deploy/tmp3503lms-vw.ear-contents/APP-INF/lib/wsdl4j-1.5.1.jar
1cef7000-1cefd000 r-xs 00000000 08:02 1966129 /var/lms/lms-server/tmp/deploy/tmp3503lms-vw.ear-contents/APP-INF/lib/vw-wsdl.jar
1cefd000-1d2a4000 r-xs 00000000 08:02 1966128 /var/lms/lms-server/tmp/deploy/tmp3503lms-vw.ear-contents/APP-INF/lib/star-xml.jar
[contents of ear deployment snipped]
1e2b0000-1e300000 r-xs 00000000 08:02 1966086 /var/lms/lms-server/tmp/deploy/tmp3503lms-vw.ear-contents/APP-INF/lib/cglib-nodep-2.1.jar
1e300000-1e4f4000 rwxp 00136000 00:00 0
1e4f4000-1e500000 ---p 00000000 00:00 0
1e500000-1ea00000 rwxp 00024000 00:00 0
1ea47000-1ea4f000 r-xs 00000000 08:02 1966085 /var/lms/lms-server/tmp/deploy/tmp3503lms-vw.ear-contents/APP-INF/lib/bzip2.jar
1ea4f000-1ebd6000 r-xs 00000000 08:02 1966084 /var/lms/lms-server/tmp/deploy/tmp3503lms-vw.ear-contents/APP-INF/lib/axis.jar
1ebd6000-1ff00000 r-xs 00000000 08:02 1835046 /var/lms/lms-server/tmp/deploy/tmp3503lms-vw.ear
1ff00000-1ffff000 rwxp 00021000 00:00 0
1ffff000-20000000 ---p 0004a000 00:00 0
2007f000-20082000 ---p 00000000 00:00 0
20082000-202fc000 rwxp 00003000 00:00 0
202fc000-20300000 ---p 00000000 00:00 0
20300000-207fe000 rwxp 00000000 00:00 0
207fe000-20800000 ---p 00000000 00:00 0
2087f000-20882000 ---p 00000000 00:00 0
20882000-20af4000 rwxp 00003000 00:00 0
20af4000-20b00000 ---p 00000000 00:00 0
20b00000-20bfc000 rwxp 00000000 00:00 0
20bfc000-20c00000 ---p 00000000 00:00 0
20c00000-20ef8000 rwxp 00004000 00:00 0
20ef8000-20f00000 ---p 00000000 00:00 0
20f00000-211ff000 rwxp 00000000 00:00 0
211ff000-21200000 ---p 00051000 00:00 0
21207000-2120a000 ---p 00000000 00:00 0
2120a000-21288000 rwxp 00003000 00:00 0
21288000-212f2000 r-xs 00000000 08:02 1966083 /var/lms/lms-server/tmp/deploy/tmp3503lms-vw.ear-contents/APP-INF/lib/antlr-2.7.5H3.jar
212f2000-21300000 r-xs 00000000 08:02 1966082 /var/lms/lms-server/tmp/deploy/tmp3503lms-vw.ear-contents/APP-INF/lib/activation.jar
21300000-213fb000 rwxp 00080000 00:00 0
213fb000-21400000 ---p 000ba000 00:00 0
21477000-214c8000 r-xs 00000000 08:02 1835030 /var/lms/lms-server/tmp/deploy/tmp3487jcommon.jar
214c8000-214ec000 r-xs 00000000 08:02 1835029 /var/lms/lms-server/tmp/deploy/tmp3486console-mgr-classes.jar
21517000-2151a000 ---p 00000000 00:00 0
2151a000-21598000 rwxp 00003000 00:00 0
21598000-215a0000 r-xs 00000000 08:02 1966124 /var/lms/lms-server/tmp/deploy/tmp3503lms-vw.ear-contents/APP-INF/lib/relaxngDatatype.jar
215a0000-21615000 r-xs 00000000 08:02 1966131 /var/lms/lms-server/tmp/deploy/tmp3503lms-vw.ear-contents/APP-INF/lib/xsdlib.jar
21657000-2173a000 r-xs 00000000 08:02 1835031 /var/lms/lms-server/tmp/deploy/tmp3488jfreechart.jar
217af000-217b2000 r-xs 00000000 08:02 1835044 /var/lms/lms-server/tmp/deploy/tmp3501mail-ra.rar
217b2000-217b6000 r-xs 00000000 08:02 1900546 /var/lms/lms-server/tmp/deploy/tmp3500jboss-xa-jdbc.rar-contents/jboss-xa-jdbc.jar
21800000-218fb000 rwxp 00049000 00:00 0
218fb000-21900000 ---p 00000000 00:00 0
21928000-2192a000 r-xs 00000000 08:02 1835040 /var/lms/lms-server/tmp/deploy/tmp3497jboss-ha-local-jdbc.rar
2197f000-21982000 ---p 00000000 00:00 0
21982000-21afa000 rwxp 00003000 00:00 0
21afa000-21b00000 ---p 000e9000 00:00 0
21b40000-21b43000 ---p 00000000 00:00 0
21b43000-21bc1000 rwxp 00003000 00:00 0
21bc1000-21c20000 r-xs 00000000 08:02 1835018 /var/lms/lms-server/tmp/deploy/tmp3475jasper-compiler.jar
21c20000-21c23000 ---p 0007c000 00:00 0
21c23000-21ca1000 rwxp 0007f000 00:00 0
21ca1000-21ca4000 ---p 00000000 00:00 0
21ca4000-21d22000 rwxp 00003000 00:00 0
21d47000-21e29000 r-xs 00000000 08:02 1835017 /var/lms/lms-server/tmp/deploy/tmp3474jasper-compiler-jdt.jar
21e29000-21e2c000 ---p 000a8000 00:00 0
21e2c000-21eaa000 rwxp 000ab000 00:00 0
21eaa000-21ead000 ---p 00129000 00:00 0
21ead000-21f2b000 rwxp 0012c000 00:00 0
21f88000-21f8f000 r-xs 00000000 08:02 2146306 /var/lms/lms-server/tmp/deploy/tmp3503lms-vw.ear-contents/lms.war/WEB-INF/lib/taglibs-response.jar
21f8f000-21fb8000 r-xs 00000000 08:02 1835028 /var/lms/lms-server/tmp/deploy/tmp3485tomcat55-service.jar
21fb8000-21ff5000 r-xs 00000000 08:02 1835027 /var/lms/lms-server/tmp/deploy/tmp3484tomcat-util.jar
21ff5000-22002000 r-xs 00000000 08:02 1835026 /var/lms/lms-server/tmp/deploy/tmp3483tomcat-http.jar
22002000-2209f000 r-xs 00000000 08:02 1835014 /var/lms/lms-server/tmp/deploy/tmp3471catalina.jar
2209f000-220a2000 ---p 00000000 00:00 0
220a2000-22120000 rwxp 00003000 00:00 0
22120000-22123000 ---p 0021a000 00:00 0
22123000-221a1000 rwxp 0021d000 00:00 0
221a1000-221a4000 ---p 0029b000 00:00 0
221a4000-22222000 rwxp 0029e000 00:00 0
22222000-22225000 ---p 0031c000 00:00 0
22225000-222a3000 rwxp 0031f000 00:00 0
222a8000-222ad000 r-xs 00000000 08:02 1835025 /var/lms/lms-server/tmp/deploy/tmp3482tomcat-coyote.jar
222ad000-222cc000 r-xs 00000000 08:02 1835024 /var/lms/lms-server/tmp/deploy/tmp3481tomcat-ajp.jar
222cc000-222d2000 r-xs 00000000 08:02 1835023 /var/lms/lms-server/tmp/deploy/tmp3480servlets-webdav.jar
222d2000-222d5000 ---p 00000000 00:00 0
222d5000-22353000 rwxp 00003000 00:00 0
22353000-22356000 ---p 0008d000 00:00 0
22356000-223d4000 rwxp 00090000 00:00 0
223d4000-223d7000 ---p 00000000 00:00 0
223d7000-22455000 rwxp 00003000 00:00 0
22459000-2245b000 r-xs 00000000 08:02 1835022 /var/lms/lms-server/tmp/deploy/tmp3479servlets-invoker.jar
2245b000-22460000 r-xs 00000000 08:02 1835021 /var/lms/lms-server/tmp/deploy/tmp3478servlets-default.jar
22460000-2246c000 r-xs 00000000 08:02 1835020 /var/lms/lms-server/tmp/deploy/tmp3477naming-resources.jar
2246c000-2247f000 r-xs 00000000 08:02 1835019 /var/lms/lms-server/tmp/deploy/tmp3476jasper-runtime.jar
2247f000-2248e000 r-xp 00000000 08:05 480101 /lib/libresolv-2.3.2.so
2248e000-2248f000 rwxp 0000f000 08:05 480101 /lib/libresolv-2.3.2.so
2248f000-22491000 rwxp 00000000 00:00 0
22491000-22495000 r-xp 00000000 08:05 480086 /lib/libnss_dns-2.3.2.so
22495000-22496000 rwxp 00003000 08:05 480086 /lib/libnss_dns-2.3.2.so
22499000-2249c000 r-xs 00000000 08:02 1916930 /var/lms/lms-server/tmp/deploy/tmp3501mail-ra.rar-contents/mail-ra.jar
2249c000-224a0000 r-xs 00000000 08:02 1835043 /var/lms/lms-server/tmp/deploy/tmp3500jboss-xa-jdbc.rar
[again contents of ear file snipped]
22bba000-22bc8000 r-xs 00000000 08:02 5668866 /var/lms/lms-server/lib/activation.jar
22bc8000-22bca000 r-xs 00000000 08:02 5668867 /var/lms/lms-server/lib/autonumber-plugin.jar
22bca000-22bd0000 r-xp 00000000 08:03 576018 /opt/jdk1.5.0_06/jre/lib/i386/libmanagement.so
22bd0000-22bd1000 rwxp 00005000 08:03 576018 /opt/jdk1.5.0_06/jre/lib/i386/libmanagement.so
22bd1000-22fec000 rwxp 00001000 00:00 0
22fec000-23000000 r-xp 00000000 08:03 576016 /opt/jdk1.5.0_06/jre/lib/i386/libnet.so
23000000-23001000 rwxp 00013000 08:03 576016 /opt/jdk1.5.0_06/jre/lib/i386/libnet.so
23001000-23002000 r-xs 00000000 08:03 64248 /opt/jboss-4.0.2/lib/namespace.jar
23002000-2303b000 r-xs 00000000 08:03 64246 /opt/jboss-4.0.2/lib/jboss-system.jar
2303b000-230ae000 r-xs 00000000 08:03 64244 /opt/jboss-4.0.2/lib/jboss-common.jar
230ae000-230cc000 r-xs 00000000 08:03 64247 /opt/jboss-4.0.2/lib/log4j-boot.jar
230cc000-230f7000 r-xs 00000000 08:03 64235 /opt/jboss-4.0.2/lib/concurrent.jar
230f7000-230ff000 r-xs 00000000 08:03 64242 /opt/jboss-4.0.2/lib/gnu-regexp.jar
230ff000-2313a000 r-xs 00000000 08:03 64243 /opt/jboss-4.0.2/lib/jaxen.jar
2313a000-231a0000 r-xs 00000000 08:03 64236 /opt/jboss-4.0.2/lib/dom4j.jar
231a0000-23231000 r-xs 00000000 08:03 64245 /opt/jboss-4.0.2/lib/jboss-jmx.jar
23231000-238e6000 r-xs 00000000 08:03 528238 /opt/jdk1.5.0_06/lib/tools.jar
238e6000-238e9000 ---p 00000000 00:00 0
238e9000-23967000 rwxp 00003000 00:00 0
23967000-23970000 r-xs 00000000 08:03 304113 /opt/jboss-4.0.2/bin/run.jar
23970000-23a34000 r-xs 00000000 08:03 16236 /opt/jdk1.5.0_06/jre/lib/ext/localedata.jar
23a34000-23a36000 r-xs 00000000 08:03 16170 /opt/jdk1.5.0_06/jre/lib/ext/dnsns.jar
23a36000-23a61000 r-xs 00000000 08:03 16169 /opt/jdk1.5.0_06/jre/lib/ext/sunpkcs11.jar
23a61000-23a87000 r-xs 00000000 08:03 16168 /opt/jdk1.5.0_06/jre/lib/ext/sunjce_provider.jar
23a87000-23a88000 ---p 00000000 00:00 0
23a88000-23b08000 rwxp 00001000 00:00 0
23b08000-23b0b000 ---p 00081000 00:00 0
23b0b000-23b89000 rwxp 00084000 00:00 0
23b89000-23b8c000 ---p 00000000 00:00 0
23b8c000-23c0a000 rwxp 00003000 00:00 0
23c0a000-23c0d000 ---p 00000000 00:00 0
23c0d000-23c8b000 rwxp 00003000 00:00 0
23c8b000-23c8e000 ---p 00000000 00:00 0
23c8e000-23d0c000 rwxp 00003000 00:00 0
23d0c000-23d0f000 ---p 00081000 00:00 0
23d0f000-23d8d000 rwxp 00084000 00:00 0
23d8d000-23d8e000 r-xp 00594000 08:05 416216 /usr/lib/locale/locale-archive
23d8e000-23e9a000 r-xp 00261000 08:05 416216 /usr/lib/locale/locale-archive
23e9a000-2409a000 r-xp 00000000 08:05 416216 /usr/lib/locale/locale-archive
2409a000-2409d000 ---p 00000000 00:00 0
2409d000-2411b000 rwxp 00003000 00:00 0
2411b000-2411e000 ---p 00081000 00:00 0
2411e000-2419c000 rwxp 00084000 00:00 0
2419c000-2419d000 ---p 00000000 00:00 0
2419d000-2422f000 rwxp 00001000 00:00 0
2422f000-2429e000 rwxp 00012000 00:00 0
2429e000-2437d000 rwxp 00000000 00:00 0
2437d000-24617000 rwxp 000df000 00:00 0
24617000-24633000 rwxp 00000000 00:00 0
24633000-24686000 rwxp 0001c000 00:00 0
24686000-24765000 rwxp 00000000 00:00 0
24765000-249ff000 rwxp 0014e000 00:00 0
249ff000-24a11000 rwxp 00000000 00:00 0
24a11000-24a7f000 rwxp 003fa000 00:00 0
24a7f000-28230000 rwxp 00000000 00:00 0
28230000-328b0000 rwxp 03c19000 00:00 0
328b0000-4e530000 rwxp 00000000 00:00 0
4e530000-a1a80000 rwxp 29f19000 00:00 0
a1a80000-a3e40000 rwxp 00000000 00:00 0
a3e40000-b1a80000 rwxp 7f829000 00:00 0
b1a81000-b1a83000 r-xs 00000000 08:02 1835041 /var/lms/lms-server/tmp/deploy/tmp3498jboss-ha-xa-jdbc.rar
b1a83000-b1a85000 r-xs 00000000 08:02 1851394 /var/lms/lms-server/tmp/deploy/tmp3497jboss-ha-local-jdbc.rar-contents/jboss-ha-local-jdbc.jar
b1a85000-b1aa1000 rwxp 00001000 00:00 0
b1aa1000-b1b05000 rwxp 0001c000 00:00 0
b1b05000-b21e5000 rwxp 00000000 00:00 0
b21e5000-b3b05000 rwxp 006e0000 00:00 0
b3b05000-b4340000 r-xs 00000000 08:03 576211 /opt/jdk1.5.0_06/jre/lib/charsets.jar
b4340000-b4354000 r-xs 00000000 08:03 576175 /opt/jdk1.5.0_06/jre/lib/jce.jar
b4354000-b43d9000 r-xs 00000000 08:03 576208 /opt/jdk1.5.0_06/jre/lib/jsse.jar
b43d9000-b4442000 rwxp 00000000 00:00 0
b4442000-b6a2a000 r-xs 00000000 08:03 576234 /opt/jdk1.5.0_06/jre/lib/rt.jar
b6a2a000-b6a3d000 r-xp 00000000 08:03 576013 /opt/jdk1.5.0_06/jre/lib/i386/libzip.so
b6a3d000-b6a3f000 rwxp 00012000 08:03 576013 /opt/jdk1.5.0_06/jre/lib/i386/libzip.so
b6a3f000-b6a60000 r-xp 00000000 08:03 576011 /opt/jdk1.5.0_06/jre/lib/i386/libjava.so
b6a60000-b6a62000 rwxp 00020000 08:03 576011 /opt/jdk1.5.0_06/jre/lib/i386/libjava.so
b6a62000-b6a6d000 r-xp 00000000 08:03 576010 /opt/jdk1.5.0_06/jre/lib/i386/libverify.so
b6a6d000-b6a6e000 rwxp 0000b000 08:03 576010 /opt/jdk1.5.0_06/jre/lib/i386/libverify.so
b6a6e000-b6a76000 rwxs 00000000 08:05 416097 /tmp/hsperfdata_jboss/27543
b6a76000-b6a81000 r-xp 00000000 08:05 480089 /lib/libnss_files-2.3.2.so
b6a81000-b6a82000 rwxp 0000a000 08:05 480089 /lib/libnss_files-2.3.2.so
b6a82000-b6a94000 r-xp 00000000 08:05 480073 /lib/libnsl-2.3.2.so
b6a94000-b6a95000 rwxp 00011000 08:05 480073 /lib/libnsl-2.3.2.so
b6a95000-b6a97000 rwxp 00000000 00:00 0
b6a9e000-b6aa4000 r-xp 00000000 08:03 496003 /opt/jdk1.5.0_06/jre/lib/i386/native_threads/libhpi.so
b6aa4000-b6aa5000 rwxp 00006000 08:03 496003 /opt/jdk1.5.0_06/jre/lib/i386/native_threads/libhpi.so
b6aa5000-b6ac6000 r-xp 00000000 08:05 192667 /lib/tls/libm-2.3.2.so
b6ac6000-b6ac7000 rwxp 00021000 08:05 192667 /lib/tls/libm-2.3.2.so
b6acc000-b6acd000 rwxp 00001000 00:00 0
b6acd000-b6ace000 r-xp 00000000 00:00 0
b6ace000-b701b000 r-xp 00000000 08:03 576004 /opt/jdk1.5.0_06/jre/lib/i386/server/libjvm.so
b701b000-b707d000 rwxp 0054d000 08:03 576004 /opt/jdk1.5.0_06/jre/lib/i386/server/libjvm.so
b707d000-b7495000 rwxp 00000000 00:00 0
b7495000-b75c8000 r-xp 00000000 08:05 192101 /