Oracle Database Discussions

Hi,

I found ORA-600 error in my production alert_log file, on further investigating i found several deadlocks and PMON not able to clean them up. I have few Questions, Is there any way we can avoid these deadlocks. Why does PMON doesn't clean them up? Can anyone suggest?

A sample of error's generated in trace log are as follows:

kjpcre: waiting for dead process 21273 cleanup by pmon
*** 2007-09-24 16:19:30.429
kjpcre: waited 60 times for dead process cleanup
*** 2007-09-24 16:20:00.632
kjpcre: waited 120 times for dead process cleanup
*** 2007-09-24 16:20:30.866
kjpcre: waited 180 times for dead process cleanup
*** 2007-09-24 16:21:01.099
kjpcre: waited 240 times for dead process cleanup
*** 2007-09-24 16:21:31.303
kjpcre: waited 300 times for dead process cleanup
*** 2007-09-24 16:22:01.526
kjpcre: waited 360 times for dead process cleanup
*** 2007-09-24 16:22:31.720
kjpcre: waited 420 times for dead process cleanup
*** 2007-09-24 16:23:01.944
kjpcre: waited 480 times for dead process cleanup
*** 2007-09-24 16:23:32.217
kjpcre: waited 540 times for dead process cleanup
*** 2007-09-24 16:24:02.431
kjpcre: waited 600 times for dead process cleanup
*** 2007-09-24 16:24:32.644
kjpcre: waited 660 times for dead process cleanup
*** 2007-09-24 16:25:02.868
kjpcre: waited 720 times for dead process cleanup
*** 2007-09-24 16:25:33.131
kjpcre: waited 780 times for dead process cleanup
*** 2007-09-24 16:26:03.345
kjpcre: waited 840 times for dead process cleanup
*** 2007-09-24 16:26:33.559
kjpcre: waited 900 times for dead process cleanup
*** 2007-09-24 16:27:03.842
kjpcre: waited 960 times for dead process cleanup
*** 2007-09-24 16:27:34.056
kjpcre: waited 1020 times for dead process cleanup
*** 2007-09-24 16:28:04.269
kjpcre: waited 1080 times for dead process cleanup
*** 2007-09-24 16:28:34.533
kjpcre: waited 1140 times for dead process cleanup
*** 2007-09-24 16:29:04.756
kjpcre: waited 1200 times for dead process cleanup
*** 2007-09-24 16:29:35.000
kjpcre: waited 1260 times for dead process cleanup
*** 2007-09-24 16:30:05.194
kjpcre: waited 1320 times for dead process cleanup
*** 2007-09-24 16:30:35.377
kjpcre: waited 1380 times for dead process cleanup
*** 2007-09-24 16:31:05.611
kjpcre: waited 1440 times for dead process cleanup
*** 2007-09-24 16:31:35.784
kjpcre: waited 1500 times for dead process cleanup
*** 2007-09-24 16:32:06.038
kjpcre: waited 1560 times for dead process cleanup
*** 2007-09-24 16:32:36.261
kjpcre: waited 1620 times for dead process cleanup
*** 2007-09-24 16:33:06.465
kjpcre: waited 1680 times for dead process cleanup
*** 2007-09-24 16:33:36.729
kjpcre: waited 1740 times for dead process cleanup
*** 2007-09-24 16:34:06.922
kjpcre: waited 1800 times for dead process cleanup
kjpcre2: pid 2147462856 node 1 flag 0x10 ksuflag 0x1
------------process---------------------
proc version : 46616
Local node : 1
gid : 0
pid : 21273
lkp_node : 1
proc state : KJP_NORMAL
Total accesses : 18411040
Imm. accesses : 18411035
Locks on ASTQ : 0
Locks Pending AST : 0
Granted locks : 0
AST_Q :NULL
PENDING_Q :NULL
GRANTED_Q :NULL
AST_Q:
PENDING_Q:
GRANTED_Q:
ABORTING SYSTEM STATE DUMP: ANOTHER PROCESS (so = 0x49130d018) EXECUTING SYSTEM STATE DUMP
*** 2007-09-24 16:34:06.930
ksedmp: internal or fatal error
ORA-00600: internal error code, arguments: [kjpcre2], [], [], [], [], [], [], []
Current SQL information unavailable - no session.
----- Call Stack Trace -----
calling call entry argument values in hex
location type point (? means dubious value)
-------------------- -------- -------------------- ----------------------------
ksedmp()+328 CALL ksedst() 00000000B ? 000000000 ?
000000000 ? 103385918 ?
00000004A ?
FFFFFFFF7FFF6FF8 ?
kgerinv()+184 PTR_CALL 0000000000000000 0001037BB ? 1037BB000 ?
1037BBE68 ? 1037C1000 ?
000102C00 ? 000000000 ?
kgeanmfe()+36 CALL kgerinv() 1037BC0C8 ? 1038D7C18 ?
0000013C8 ? 000000000 ?
1037BE1D4 ? 1037BD498 ?
kjpcre()+472 CALL kgeanmfe() 1037BC0C8 ? 1038D7C18 ?
1033F2318 ? 000000000 ?
0000530E8 ? 000005319 ?
kjual()+808 CALL kjpcre() FFFFFFFF7FFFAEC8 ?
100000400 ? 4926F80D0 ?
1033F2318 ? 1037C1DE8 ?
1037C1DB0 ?
kjinfy()+1436 CALL kjual() 1033F0A18 ? 1037C1DB0 ?
000380000 ? 1037C1DE8 ?
1037BE798 ? 046F7B944 ?
kscnfy()+152 PTR_CALL 0000000000000000 38000C000 ? 49130B538 ?
00000000F ? 000007400 ?
000000002 ? 000000002 ?
ksucrp()+1204 CALL kscnfy() 00000000F ? 49130B538 ?
4912F3CF8 ? 000017840 ?
000000046 ? 000002800 ?
ksucresg()+36 CALL ksucrp() 000000003 ? 000000000 ?
380007BC8 ? 380008000 ?
1037BE740 ? 000380000 ?
kpolna()+316 CALL ksucresg() 000000000 ? 000000000 ?
000000000 ? 000103400 ?
0001037BE ? 1037BE000 ?
kpogsk()+108 CALL kpolna() FFFFFFFF7FFFB880 ?
000000000 ? 1037BC208 ?
1037C3120 ? 000000000 ?
000000140 ?
opiodr()+1720 PTR_CALL 0000000000000000 000000076 ? 000000007 ?
FFFFFFFF7FFFE6C8 ?
1037C4000 ? 1037C4258 ?
000103400 ?
ttcpip()+1564 PTR_CALL 0000000000000000 000102D8F ? 0001037BC ?
1037C42E8 ? 1037BC208 ?
1037BE748 ?
FFFFFFFF7FFFC070 ?
opitsk()+1156 CALL ttcpip() 1037C42E0 ? 000000001 ?
FFFFFFFF7FFFE6C8 ?
000000007 ?
FFFFFFFF7FFFD1B8 ?
FFFFFFFF7FFFD1B4 ?
opiino()+1504 CALL opitsk() 000000000 ? 1037C4270 ?
000000000 ? 000000000 ?
102DFE808 ? 1037BE730 ?
opiodr()+1720 PTR_CALL 0000000000000000 000000000 ? 000000000 ?
000000000 ?
FFFFFFFF7FFFF540 ?
000000001 ? 000000001 ?
opidrv()+764 CALL opiodr() 000102D8F ? 0001037BC ?
1037C42E8 ? 1037BC208 ?
1037BE748 ?
FFFFFFFF7FFFF060 ?
sou2o()+16 CALL opidrv() 0001037BB ? 000000004 ?
1037BC208 ? 00000003C ?
1037BC208 ? 1037BBE68 ?
main()+184 CALL sou2o() FFFFFFFF7FFFF560 ?
00000003C ? 000000004 ?
FFFFFFFF7FFFF540 ?
00003B400 ? 00003B678 ?
_start()+380 CALL main() 000000002 ?
FFFFFFFF7FFFF6A8 ?
FFFFFFFF7FFFF6C0 ?
000000000 ?
FFFFFFFF7DE00100 ?
FFFFFFFF7DE00140 ?

--------------------- Binary Stack Dump ---------------------

========== FRAME [1] (ksedmp()+328 -> ksedst()) ==========
%l0 00000001037C1034 %l1 0000000000000000 %l2 FFFFFFFF7FFF2AB0
%l3 0000000000000000 %l4 000000000000000C %l5 FFFFFFFF7FFF6698
%l6 FFFFFFFF7FFF66F8 %l7 FFFFFFFF7FFF6FF8 %i0 000000000000004C
%i1 0000000000000000 %i2 0000000000000000 %i3 0000000103385918
%i4 0000000000000000 %i5 FFFFFFFF7FFF6FF8 %fp FFFFFFFF7FFF9D51
rtn-pc 0000000100587848 argd FFFFFFFF7FFF2960 stret FFFFFFFF7FFF2960
xtraarg FFFFFFFF7FFF9E01 locals FFFFFFFF7FFF2998
Dump of memory from 0xFFFFFFFF7FFF28E0 to 0xFFFFFFFF7FFF2CE0
FFFFFFFF7FFF28E0 00000001 037C1000 00000001 037C1034 [.....|.......|.4]
FFFFFFFF7FFF28F0 00000000 00000000 00000000 001037C1 [..............7.]
FFFFFFFF7FFF2900 00000000 00000008 FFFFFFFF 7FFF6698 [..............f.]
FFFFFFFF7FFF2910 00000000 00103000 00000000 00103385 [......0.......3.]
FFFFFFFF7FFF2920 00000000 00000072 00000000 00000000 [.......r........]
FFFFFFFF7FFF2930 00000000 00000118 00000001 03385918 [.............8Y.]
FFFFFFFF7FFF2940 00000001 03385988 FFFFFFFF 7FFF70A0 [.....8Y.......p.]
FFFFFFFF7FFF2950 FFFFFFFF 7FFF9D51 00000001 00587848 [.......Q.....XxH]
FFFFFFFF7FFF2960 00000000 00000001 FFFFFFFF 7FFF5D50 [..............]P]
FFFFFFFF7FFF2970 FFFFFFFF 7FFFA509 00000000 00000006 [................]
FFFFFFFF7FFF2980 00000000 00000006 00000001 03385930 [.............8Y0]

An input would be much appreciated.

Regards,

ODBA