JVM Crashes Sporadically with ORA-24550 while Running Tomcat Web App
We have experienced a problem that has been troubling us for some time now. Our website goes down sporadically as a result of a JVM crash. The Tomcat catalina.out log reads as follows:
# A fatal error has been detected by the Java Runtime Environment:
#
# SIGSEGV (0xb) at pc=0x00002aaac5938c2b, pid=2214, tid=47766235233056
#
# JRE version: 7.0_07-b10
# Java VM: Java HotSpot(TM) 64-Bit Server VM (23.3-b01 mixed mode linux-amd64 compressed oops)
# Problematic frame:
# C [libclntsh.so.11.1+0x15f7c2b] nnfltimexp+0x1f
#
# Failed to write core dump. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
#
# An error report file with more information is saved as:
# /usr/local/tomcat/tomcat-gbco2/logs/hs_err_pid2214.log
#
# If you would like to submit a bug report, please visit:
# http://bugreport.sun.com/bugreport/crash.jsp
#
ORA-24550: signal received: [si_signo=6] [si_errno=0] [si_code=-6] [si_int=367714944] [si_ptr=0xffff810115eae280] [si_addr=0x69000008a6]
kpedbg_dmp_stack()+360<-kpeDbgCrash()+192<-kpeDbgSignalHandler()+119<-skgesig_sigactionHandler()+218<-__sighandler()<-gsignal()+53<-00002B71717BC1F0<-00002B71717BC1F0<-00002B71717BC1F0<-00002B71717BC1F0<-00002B71717BC1F0<-00002B71717BC1F0<-00002B71717BC1F0<-00002B71717BC1F0<-00002B71717BC1F0<-00002B71717BC1F0<-00002B71717BC1F0<-00002B71717BC1F0<-00002B71717BC1F0<-00002B71717BC1F0<-00002B71717BC1F0<-00002B71717BC1F0<-00002B71717BC1F0<-00002B71717BC1F0<-00002B71717BC1F0
I am no expert in reading crash logs, but I can provide it if it would be helpful.
We are running our app on
Intel (R) Xeon (R) CPU X5550 @ 2.67GHz with 4 cpu cores.
CentOS Release 5.7 (Linux xena.pcbi.upenn.edu 2.6.18-274.3.1.el5 #1 SMP Tue Sep 6 20:13:52 EDT 2011 x86_64 x86_64 x86_64 GNU/Linux)
Total server RAM: 24GB
Tomcat 6.0.26
Java 7.0_07-b10 (Java VM: Java HotSpot(TM) 64-Bit Server VM)
Oracle Enterprise database server (11.2.0.3) - (separate server)
OCI 11.2.0.3 (64-bit client)
Our sqlnet.ora file contains the following:
NAMES.DIRECTORY_PATH= (LDAP, TNSNAMES, EZCONNECT)
DIAG_ADR_ENABLED=OFF
DIAG_SIGHANDLER_ENABLED=FALSE
DIAG_DDE_ENABLED=FALSE
We read about issues with OCI 11.2.0.1 in the Oracle support website and we upgraded. But the issue remains. We are at a loss as to where to look/what to do next. Any suggestions would indeed be most welcome.
Many thanks -
Cris