Skip to Main Content

Oracle Database Discussions

Announcement

For appeals, questions and feedback about Oracle Forums, please email oracle-forums-moderators_us@oracle.com. Technical questions should be asked in the appropriate category. Thank you!

How unhealthy is this RAC?

steffiMar 20 2011 — edited Mar 20 2011
Here's is the contents of v$system_event..

Is this
EVENT	TOTAL_WAITS	TIME_WAITED	AVERAGE_WAIT
enq: TX - index contention	40564851	214701526	 5.29
enq: TX - row lock contention	188846	12454614	65.95
enq: SQ - contention	141971	70568	0.5

cause for concern? 

EVENT	TOTAL_WAITS	TIME_WAITED	AVERAGE_WAIT
SQL*Net message to client	6015051449	607254	0
SQL*Net message from client	6015048542	178177969892	29.62
gcs remote message	2948555287	2633481757	0.89
CGS wait for IPC msg	1517805027	634397	0
db file sequential read	1500615188	816364485	0.54
ges remote message	1247679701	1407300224	1.13
gc cr multi block request	778432813	9913464	0.01
gc current block 2-way	747852637	38030616	0.05
db file scattered read	709428365	460939295	0.65
rdbms ipc message	708473316	37650068633	53.14
gc buffer busy acquire	671285134	1033621285	1.54
PX Deq: reap credit	667784615	484449	0
gcs log flush sync	592376026	171712257	0.29
gc cr block 2-way	530861847	19607062	0.04
library cache pin	437937120	15126237	0.03
log file sync	379523272	797193932	2.1
DIAG idle wait	359607166	2822108755	7.85
log file parallel write	351225436	259263769	0.74
LNS ASYNC end of log	350170653	1398410516	3.99
LNS wait on SENDREQ	321652621	3209301	0.01
PX qref latch	297396661	94308	0
read by other session	289140108	148440270	0.51
buffer deadlock	163505781	983055	0.01
gc current block busy	119223825	467716658	3.92
PX Deq: Table Q Normal	117332841	23574867	0.2
ksxr poll remote instances	110480324	90333	0
buffer busy waits	106938153	19933900	0.19
direct path read	93429599	108427028	1.16
SQL*Net more data from client	86471785	23026529	0.27
gc current grant busy	84978157	28215346	0.33
control file sequential read	82646297	23694583	0.29
PX Deq Credit: send blkd	78641669	9569299	0.12
latch: cache buffers chains	74218671	690277	0.01
gc current grant 2-way	72557796	1920419	0.03
library cache: mutex X	71106697	75993	0
DFS lock handle	70722498	2716407	0.04
gc cr grant 2-way	64558237	1633004	0.03
PX Deq: Execution Msg	61706261	314222076	5.09
gc cr block busy	61469863	119850802	1.95
library cache lock	52428649	3773354	0.07
PX Deq: Slave Session Stats	48040224	1886805	0.04
db file parallel read	46415188	118467902	2.55
IPC send completion sync	46250594	965101	0.02
enq: TX - index contention	40564851	214701526	5.29
PX Deq: Execute Reply	39689685	17243721	0.43
gc buffer busy release	36976909	242714774	6.56
SQL*Net more data to client	36627952	44167	0
PX Deq: Msg Fragment	30501244	343397	0.01
rdbms ipc reply	29725302	1352370	0.05
RMAN backup & recovery I/O	28824547	37722662	1.31
reliable message	27892263	3082134	0.11
PX Idle Wait	27356097	4651277341	170.03
ASM file metadata operation	25098749	8850323	0.35
gc object scan	22705857	7485	0
db file parallel write	19896252	52152606	2.62
latch: ges resource hash list	19336183	427451	0.02
enq: PS - contention	19143961	707455	0.04
PX Deq: Parse Reply	19093356	895799	0.05
gc cr disk read	17816846	448909	0.03
ASM background timer	16101806	1383957874	85.95
PX Deq: Slave Join Frag	16044789	233149	0.01
wait for unread message on broadcast channel	15056320	1413552546	93.88
cursor: mutex X	13435193	24140	0
KJC: Wait for msg sends to complete	13268497	11397	0
PX Deq: Signal ACK RSG	13214824	101941	0.01
KSV master wait	13206286	4235645	0.32
direct path read temp	12617694	5487608	0.43
PX Deq Credit: need buffer	11675868	879967	0.08
row cache lock	11536185	398216	0.03
PX Deq Credit: Session Stats	9480862	78910	0.01
SQL*Net message to dblink	9312894	1538	0
SQL*Net message from dblink	9312894	6279631	0.67
control file parallel write	7760982	11854435	1.53
pmon timer	7558889	1412576090	186.88
PX Deq: Join ACK	7548816	498931	0.07
gc current multi block request	6035173	155898	0.03
PING	5706961	1413230267	247.63
enq: XR - database force logging	4662671	198813	0.04
class slave wait	4561877	7097429006	1555.81
Streams AQ: waiting for messages in the queue	4495828	1543411682	343.3
SQL*Net more data from dblink	3696582	444575	0.12
LGWR wait for redo copy	3655353	17840	0
log file sequential read	3387305	6610414	1.95
Log archive I/O	2990486	276772	0.09
SQL*Net break/reset to client	2971976	2385935	0.8
direct path write temp	2839390	2522114	0.89
Space Manager: slave idle wait	2827526	1412987186	499.73
latch: shared pool	2808517	298150	0.11
latch: gc element	2421717	24688	0.01
SGA: MMAN sleep for component shrink	2336447	2458094	1.05
latch: enqueue hash chains	2279645	15435	0.01
latch free	2089418	78732	0.04
gc current split	2044784	1864009	0.91
PX Deq: Signal ACK EXT	1976164	19263	0.01
enq: FB - contention	1473469	61036	0.04
cursor: pin S wait on X	1313129	1464789	1.12
SQL*Net more data to dblink	1232891	986	0
Streams AQ: RAC qmn coordinator idle wait	1211300	788	0
enq: HW - contention	1175390	2077008	1.77
latch: session allocation	1167768	21883	0.02
Streams AQ: qmn coordinator idle wait	1144699	1412546634	1233.99
Streams AQ: qmn slave idle wait	1031585	2227183681	2158.99
lock deadlock retry	962937	4698	0
enq: CF - contention	956154	609647	0.64
latch: cache buffers lru chain	902764	37552	0.04
latch: object queue header operation	817911	27717	0.03
global enqueue expand wait	768633	654105	0.85
Data file init write	756191	329758	0.44
latch: gcs resource hash	647021	4147	0.01
local write wait	603007	286191	0.47
latch: row cache objects	599358	6453	0.01
ges lmd/lmses to freeze in rcfg - mrcvr	481759	156345	0.32
shared server idle wait	471190	1413238589	2999.3
enq: RF - DG Broker Current File ID	469833	23209	0.05
smon timer	432383	1411851085	3265.28
SGA: allocation forcing component growth	363333	379008	1.04
gc current retry	341104	1121252	3.29
enq: RF - synch: DG Broker metadata	319143	588290	1.84
enq: PG - contention	313659	14830	0.05
enq: TT - contention	260134	11207172	43.08
enq: KO - fast object checkpoint	236745	820808	3.47
dispatcher timer	236637	1413242481	5972.2
direct path write	231382	191008	0.83
cursor: pin S	229011	394	0
Streams AQ: waiting for time management or cleanup tasks	199981	1413148548	7066.41
enq: TX - row lock contention	188846	12454614	65.95
enq: TX - allocate ITL entry	153703	54252	0.35
enq: SQ - contention	141971	70568	0.5
ksdxexeother	141885	56	0
latch: redo allocation	138912	1858	0.01
recovery area: computing applied logs	126415	45925	0.36
gc current block congested	126318	21768	0.17
resmgr:cpu quantum	123074	151384	1.23
jobq slave wait	120678	35574221	294.79
Datapump dump file I/O	90431	9127	0.1
ges inquiry response	89402	4041	0.05
os thread startup	83809	222586	2.66
cr request retry	80062	71896	0.9
PX Deq: Table Q Sample	79665	133402	1.67
gc cr block congested	79026	14792	0.19
gc cr failure	77521	25019	0.32
enq: WF - contention	73983	825388	11.16
enq: TQ - TM contention	72871	3319	0.05
lock escalate retry	65714	1574	0.02
buffer exterminate	59775	64919	1.09
fbar timer	47136	1413183353	29980.98
log file switch completion	46911	452097	9.64
recovery area: computing obsolete files	45699	8547	0.19
enq: US - contention	40401	8805	0.22
enq: TM - contention	39149	5435032	138.83
library cache load lock	36311	382575	10.54
kjbdrmcvtq lmon drm quiesce: ping completion	31668	47443	1.5
enq: TD - KTF dump entries	31468	1424	0.05
enq: RO - fast object reuse	28422	31772	1.12
parallel recovery slave wait for change	27558	3163	0.11
name-service call wait	23694	181533	7.66
control file single write	22375	1624	0.07
kksfbc child completion	21239	106926	5.03
PX Deq: Table Q qref	19325	245	0.01
enq: TX - contention	18805	113253	6.02
latch: messages	17203	181	0.01
enq: RS - prevent file delete	16913	1013	0.06
enq: RS - prevent aging list update	15682	642	0.04
PX Deq: Table Q Get Keys	14322	42935	3
gc current grant congested	14292	2192	0.15
cursor: mutex S	13285	8	0
log file single write	13164	5371	0.41
latch: undo global data	12649	178	0.01
kksfbc research	11894	12680	1.07
parallel recovery slave idle wait	11193	5872	0.52
wait list latch free	11026	11794	1.07
enq: CT - state	11001	417	0.04
latch: checkpoint queue latch	10526	132	0.01
enq: PE - contention	10506	1139	0.11
ARCH wait on SENDREQ	9957	216480	21.74
gc cr grant congested	9465	1584	0.17
wait for scn ack	9377	3155	0.34
enq: TA - contention	8856	324	0.04
log buffer space	8777	89323	10.18
enq: TK - Auto Task Serialization	8542	343	0.04
enq: DR - contention	7842	323	0.04
process diagnostic dump	7707	2072	0.27
JOX Jit Process Sleep	7612	11286431	1482.72
enq: TC - contention	7357	340817	46.33
ges global resource directory to be frozen	7140	12299	1.72
enq: CO - master slave det	6850	312	0.05
enq: JS - job run lock - synchronize	6704	397	0.06
gcs drm freeze in enter server mode	6542	40742	6.23
enq: TS - contention	5959	89332	14.99
ARCH wait for archivelog lock	5600	36	0.01
PX Nsq: PQ load info query	5377	104798	19.49
db file single write	5373	3452	0.64
gc remaster	5315	50625	9.52
latch: parallel query alloc buffer	4939	1906	0.39
enq: TO - contention	4799	143	0.03
enq: AF - task serialization	4395	161	0.04
enq: PI - contention	4251	163	0.04
ges2 LMON to wake up LMD - mrcvr	4210	28	0.01
enq: DL - contention	3889	239	0.06
kjctssqmg: quick message send wait	3408	22	0.01
LNS wait on DETACH	3275	741	0.23
ksfd: async disk IO	3274	1	0
LNS wait on ATTACH	3273	51940	15.87
ARCH wait on DETACH	3231	714	0.22
ARCH wait on ATTACH	3226	43238	13.4
enq: BR - file shrink	2787	116	0.04
write complete waits	2631	1070	0.41
enq: MD - contention	2596	67	0.03
enq: WL - contention	2198	266518	121.25
single-task message	2098	25896	12.34
enq: OD - Serializing DDLs	2054	66	0.03
resmgr:internal state change	2001	14735	7.36
ARCH wait on c/f tx acquire 2	1751	175230	100.07
enq: WR - contention	1636	69	0.04
latch: cache buffer handles	1610	29	0.02
statement suspended, wait error to be cleared	1497	22626	15.11
Streams AQ: qmn coordinator waiting for slave to start	1214	678966	559.28
enq: PD - contention	1182	33	0.03
JS kgl get object wait	1096	4922	4.49
undo segment extension	1070	10065	9.41
PL/SQL lock timer	949	8739819	9209.5
enq: AE - lock	937	28	0.03
LGWR-LNS wait on channel	832	913	1.1
ges DFS hang analysis phase 2 acks	816	495	0.61
latch: redo writing	729	9	0.01
gc quiesce	665	564	0.85
enq: JS - queue lock	482	2111	4.38
PX Deq: Test for credit	442	13	0.03
enq: SS - contention	386	274	0.71
recovery area: computing dropped files	328	1400	4.27
recovery area: computing backed up files	328	496	1.51
ksdxexeotherwait	279	10592	37.97
log switch/archive	250	137570	550.28
gc domain validation	223	39964	179.21
auto-sqltune: wait graph update	195	96514	494.95
wait for a undo record	170	1214	7.14
parallel recovery coord send blocked	168	4	0.02
enq: JS - wdw op	168	3741	22.27
enq: KT - contention	165	5	0.03
switch logfile command	156	6290	40.32
gcs resource directory to be unfrozen	149	12839	86.17
Data Guard Broker Wait	139	10906	78.46
enq: SK - contention	129	4	0.03
enq: JS - job recov lock	128	4	0.03
gc cr block lost	125	6772	54.17
virtual circuit wait	122	3	0.03
ges LMON to get to FTDONE 	100	187	1.87
enq: CU - contention	80	242	3.02
enq: JQ - contention	78	7	0.09
cursor: pin X	73	83	1.14
parallel recovery coord wait for reply	70	510	7.29
PX Deq: Txn Recovery Start	67	3436	51.29
SQL*Net break/reset to dblink	60	0	0
gc current block lost	57	2869	50.33
ges LMD suspend for testing event	51	709	13.89
inactive session	46	4550	98.91
recovery read	45	5	0.11
JS kill job wait	41	3548	86.53
enq: AS - service activation	40	1	0.03
enq: TL - contention	35	2	0.05
enq: UL - contention	34	524	15.42
gcs enter server mode	33	1559	47.24
wait for stopper event to be increased	30	218	7.27
enq: TQ - DDL contention	24	300	12.52
enq: MR - contention	21	1	0.03
ges reconfiguration to start	20	54	2.72
ges enter server mode	20	502	25.08
buffer latch	18	1337	74.26
enq: SR - contention	18	1	0.05
Streams: RAC waiting for inter instance ack	18	3748	208.21
enq: PR - contention	17	46	2.72
kupp process wait	16	166	10.39
checkpoint completed	15	3678	245.19
PX Deque wait	14	68	4.87
enq: BF - allocation contention	14	1	0.08
enq: XL - fault extent map	14	51	3.66
enq: FU - contention	14	17	1.18
enq: TH - metric threshold evaluation	13	114	8.78
enq: MW - contention	12	0	0.04
enq: DD - contention	10	0	0.04
process terminate	8	41	5.08
ges cgs registration	8	151	18.9
buffer resize	7	0	0
ktm: instance recovery	7	698	99.66
LNS wait on LGWR	6	0	0
ASM background starting	6	381	63.43
gc cr block 3-way	5	0	0.08
enq: PV - syncstart	5	9	1.74
Global transaction acquire instance locks	4	4	1.09
enq: RS - read alert level	4	0	0.04
LGWR wait on LNS	3	0	0
gc recovery	3	540	179.85
Streams AQ: enqueue blocked on low memory	3	544	181.2
DBWR range invalidation sync	3	17	5.83
enq: DM - contention	3	0	0.03
enq: RF - FSFO Observer Heartbeat	3	0	0.03
enq: JS - q mem clnup lck	3	0	0
DG Broker configuration file I/O	2	0	0
enq: RC - Result Cache: Contention	2	493	246.6
enq: KM - contention	2	0	0.03
enq: RT - contention	2	0	0.04
instance state change	2	0	0.12
kkdlgon	2	10	5.11
enq: TQ - INI contention	2	292	146.07
enq: JS - contention	2	0	0
ARCH wait for netserver start	1	400	400.02
log file switch (checkpoint incomplete)	1	3	3.44
JS coord start wait	1	50	50.09
ges lmd and pmon to attach	1	1	1.26
wait for tmc2 to complete	1	3	3.03
control file heartbeat	1	400	400.02
enq: SW - contention	1	0	0.04
enq: PW - perwarm status in dbw0	1	0	0.09
enq: FS - contention	1	0	0.04
enq: XR - quiesce database	1	0	0.04
enq: RS - write alert level	1	0	0.02
enq: CN - race with init	1	0	0.03
enq: FE - contention	1	4	3.77
Wait for shrink lock2	1	10	10.03
enq: IA - contention	1	0	0.02
enq: RF - atomicity	1	0	0.05
enq: RF - synchronization: aifo master	1	0	0.02
enq: RF - RF - Database Automatic Disable	1	0	0.06
enq: WP - contention	1	0	0.02
enq: TB - SQL Tuning Base Cache Load	1	0	0.05
enq: JS - evt notify	1	0	0.02
Edited by: steffi on Mar 20, 2011 12:21 AM

Edited by: steffi on Mar 20, 2011 8:18 AM

Edited by: steffi on Mar 20, 2011 8:19 AM
Comments
Locked Post
New comments cannot be posted to this locked post.
Post Details
Locked on Apr 17 2011
Added on Mar 20 2011
6 comments
6,742 views