CPU Usage 99%
Hi Folks,
Have a problem and hoping you guys can help me out -
Running Oracle 10.2.0.4 on Windows 2003 Server 32 bit
APEX application version 2.2.0.00.32
Recently we've been having a peculiar problem. The CPU usage runs at 99% after a little while of starting the database.
It starts off at 0%, gets up to 50% after 5-10 min and gets to 99% after about 15 min.
Alert log is pointing to errors in trace file in udump dir.
relevant trace file shows
ORA-00600: internal error code, arguments: [13011], [50959], [12612110], [0], [12612110], [3], [], []
Current running statement to be :
DELETE FROM MGMT_SPACE_METRICS WHERE TARGET_GUID = :B2 AND METRIC_NAME = :B1 AND COLLECTION_TIMESTAMP < (SELECT MAX(COLLECTION_TIMESTAMP) FROM MGMT_SPACE_METRICS SM WHERE SM.TARGET_GUID = :B2 AND SM.METRIC_NAME = :B1 )
In other trace files the current running statement is :
DELETE FROM "MGMT_DB_INIT_PARAMS_ECM" WHERE ECM_SNAPSHOT_ID = :B1
Also see some wait history is some of the trace files. These are the waits seen in the trace file
Dumping Session Wait History
for 'db file sequential read' count=1 wait_time=7842
file#=3, block#=c671, blocks=1
for 'log file sync' count=1 wait_time=198633
buffer#=1630, =0, =0
for 'enq: TX - row lock contention' count=1 wait_time=2995900
name|mode=54580006, usn<<16 | slot=8001b, sequence=4a89
for 'log file sync' count=1 wait_time=183002
buffer#=162e, =0, =0
for 'enq: TX - row lock contention' count=1 wait_time=2995591
name|mode=54580006, usn<<16 | slot=5002b, sequence=4ab5
for 'enq: TX - row lock contention' count=1 wait_time=2995446
name|mode=54580006, usn<<16 | slot=30021, sequence=4a55
for 'enq: TX - row lock contention' count=1 wait_time=2994713
name|mode=54580006, usn<<16 | slot=60009, sequence=4a6c
for 'enq: TX - row lock contention' count=1 wait_time=3077793
name|mode=54580006, usn<<16 | slot=20005, sequence=4a4e
for 'enq: TX - row lock contention' count=1 wait_time=3078208
name|mode=54580006, usn<<16 | slot=20005, sequence=4a4e
for 'enq: TX - row lock contention' count=1 wait_time=3124993
name|mode=54580006, usn<<16 | slot=20005, sequence=4a4e
The process (looking up the number from trace file in v$process) that throws the error is a shadow process with sysman as the connected userid.
Here are the top waits from v$system_events
EVENT TOTAL_WAITS TOTAL_TIMEOUTS
---------------------------------------------------------------- ----------- --------------
rdbms ipc message 7275 5873
wait for unread message on broadcast channel 1599 1597
jobq slave wait 676 667
pmon timer 648 645
Streams AQ: waiting for messages in the queue 369 365
PX Idle Wait 270 287
enq: TX - row lock contention 230 230
Streams AQ: qmn coordinator idle wait 131 67
virtual circuit status 62 61
dispatcher timer 30 30
Any pointers as to what the problem or fix might be ?
TIA
Norm.