OGG 18.1.0.0
Orace DB 12.1.0.2 RAC 2 nodes
OS: Solaris 11
Dear Team,
recently I noticed the extract was very slow: RBA was not moving, Checkpoint lag kept on increasing.
Extract report file shows long running transaction.
2022-02-22 16:03:43 WARNING OGG-01027 Long Running Transaction: XID 0.22.63.666888, Items 0, Extract EXTDWH, Redo Thread 1, SCN 84.221773432 (360999026296), Redo Seq #23328, Redo RBA 92428420.
2022-02-22 16:25:18 INFO OGG-01971 The previous message, 'WARNING OGG-01027', repeated 1 times.
2022-02-22 16:25:18 INFO OGG-01021 Command received from GGSCI: STATUS.
2022-02-22 16:25:48 INFO OGG-01971 The previous message, 'INFO OGG-01021', repeated 1 times.
2022-02-22 16:33:43 WARNING OGG-01027 Long Running Transaction: XID 0.22.63.666888, Items 0, Extract EXTDWH, Redo Thread 1, SCN 84.221773432 (360999026296), Redo Seq #23328, Redo RBA 92428420.
2022-02-22 16:46:48 INFO OGG-01021 Command received from GGSCI: STAUTS.
2022-02-22 16:46:51 INFO OGG-01021 Command received from GGSCI: STATUS.
2022-02-22 16:48:43 WARNING OGG-01027 Long Running Transaction: XID 0.22.63.666888, Items 0, Extract EXTDWH, Redo Thread 1, SCN 84.221773432 (360999026296), Redo Seq #23328, Redo RBA 92428420.
2022-02-22 16:49:15 INFO OGG-01021 Command received from GGSCI: STATUS.
2022-02-22 16:51:45 INFO OGG-01971 The previous message, 'INFO OGG-01021', repeated 1 times.
2022-02-22 16:03:43 WARNING OGG-01027 Long Running Transaction: XID 0.22.63.666888, Items 0, Extract EXTDWH, Redo Thread 1, SCN 84.221773432 (360999026296), Redo Seq #23328, Redo RBA 92428420.
2022-02-22 16:25:18 INFO OGG-01971 The previous message, 'WARNING OGG-01027', repeated 1 times.
2022-02-22 16:25:18 INFO OGG-01021 Command received from GGSCI: STATUS.
2022-02-22 16:25:48 INFO OGG-01971 The previous message, 'INFO OGG-01021', repeated 1 times.
2022-02-22 16:33:43 WARNING OGG-01027 Long Running Transaction: XID 0.22.63.666888, Items 0, Extract EXTDWH, Redo Thread 1, SCN 84.221773432 (360999026296), Redo Seq #23328, Redo RBA 92428420.
2022-02-22 16:46:48 INFO OGG-01021 Command received from GGSCI: STAUTS.
2022-02-22 16:46:51 INFO OGG-01021 Command received from GGSCI: STATUS.
2022-02-22 16:48:43 WARNING OGG-01027 Long Running Transaction: XID 0.22.63.666888, Items 0, Extract EXTDWH, Redo Thread 1, SCN 84.221773432 (360999026296), Redo Seq #23328, Redo RBA 92428420.
2022-02-22 16:49:15 INFO OGG-01021 Command received from GGSCI: STATUS.
2022-02-22 16:51:45 INFO OGG-01971 The previous message, 'INFO OGG-01021', repeated 1 times.
2022-02-22 16:03:43 WARNING OGG-01027 Long Running Transaction: XID 0.22.63.666888, Items 0, Extract EXTDWH, Redo Thread 1, SCN 84.221773432 (360999026296), Redo Seq #23328, Redo RBA 92428420.
2022-02-22 16:25:18 INFO OGG-01971 The previous message, 'WARNING OGG-01027', repeated 1 times.
2022-02-22 16:25:18 INFO OGG-01021 Command received from GGSCI: STATUS.
2022-02-22 16:25:48 INFO OGG-01971 The previous message, 'INFO OGG-01021', repeated 1 times.
2022-02-22 16:33:43 WARNING OGG-01027 Long Running Transaction: XID 0.22.63.666888, Items 0, Extract EXTDWH, Redo Thread 1, SCN 84.221773432 (360999026296), Redo Seq #23328, Redo RBA 92428420.
2022-02-22 16:46:48 INFO OGG-01021 Command received from GGSCI: STAUTS.
2022-02-22 16:46:51 INFO OGG-01021 Command received from GGSCI: STATUS.
2022-02-22 16:48:43 WARNING OGG-01027 Long Running Transaction: XID 0.22.63.666888, Items 0, Extract EXTDWH, Redo Thread 1, SCN 84.221773432 (360999026296), Redo Seq #23328, Redo RBA 92428420.
2022-02-22 16:49:15 INFO OGG-01021 Command received from GGSCI: STATUS.
2022-02-22 16:51:45 INFO OGG-01971 The previous message, 'INFO OGG-01021', repeated 1 times.
2022-02-22 16:03:43 WARNING OGG-01027 Long Running Transaction: XID 0.22.63.666888, Items 0, Extract EXTDWH, Redo Thread 1, SCN 84.221773432 (360999026296), Redo Seq #23328, Redo RBA 92428420.
2022-02-22 16:25:18 INFO OGG-01971 The previous message, 'WARNING OGG-01027', repeated 1 times.
2022-02-22 16:25:18 INFO OGG-01021 Command received from GGSCI: STATUS.
2022-02-22 16:25:48 INFO OGG-01971 The previous message, 'INFO OGG-01021', repeated 1 times.
2022-02-22 16:33:43 WARNING OGG-01027 Long Running Transaction: XID 0.22.63.666888, Items 0, Extract EXTDWH, Redo Thread 1, SCN 84.221773432 (360999026296), Redo Seq #23328, Redo RBA 92428420.
2022-02-22 16:46:48 INFO OGG-01021 Command received from GGSCI: STAUTS.
2022-02-22 16:46:51 INFO OGG-01021 Command received from GGSCI: STATUS.
2022-02-22 16:48:43 WARNING OGG-01027 Long Running Transaction: XID 0.22.63.666888, Items 0, Extract EXTDWH, Redo Thread 1, SCN 84.221773432 (360999026296), Redo Seq #23328, Redo RBA 92428420.
Extract status:
GGSCI (dware1) 2> send EXTDWH status
Sending STATUS request to EXTRACT EXTDWH ...
EXTRACT EXTDWH (PID 14915)
Current status: Recovery complete: Processing data
Current read position:
Redo thread #: 1
Sequence #: 23328
RBA: 104262272
Timestamp: 2022-02-22 15:46:18.000000
SCN: 84.221785392 (360999038256)
Current write position:
Sequence #: 14157
RBA: 196854618
Timestamp: 2022-02-22 17:00:07.981727
Extract Trail: /ogg/ora12c/dirdat/aa
2022-02-22 16:49:15 INFO OGG-01021 Command received from GGSCI: STATUS.
LOGDUMP OUTPUT:
2022-02-22 16:51:45 INFO OGG-01971 The previous message, 'INFO OGG-01021', repeated 1 times.
Logdump 816 >open /ogg/ora12c/dirdat/aa000014156
Current LogTrail is /ogg/ora12c/dirdat/aa000014156
Logdump 817 >ghdr on
Logdump 818 >pos 196854618
Reading forward from RBA 196854618
Logdump 819 >n
Bad record found at RBA 196854618, format 5.50 Header token)
3f00 0a00 | ?...
Logdump 820 >ggstoken details on
Logdump 821 >n
Bad record found at RBA 196854618, format 5.50 Header token)
3f00 0a00 | ?...
Logdump 822
GGSCI (dware1) 2> send EXTDWH status
Sending STATUS request to EXTRACT EXTDWH ...
EXTRACT EXTDWH (PID 14915)
Current status: Recovery complete: Processing data
Current read position:
Redo thread #: 1
Sequence #: 23328
RBA: 104262272
Timestamp: 2022-02-22 15:46:18.000000
SCN: 84.221785392 (360999038256)
Current write position:
Sequence #: 14157
RBA: 196854618
Timestamp: 2022-02-22 17:00:07.981727
Extract Trail: /ogg/ora12c/dirdat/aa
When I checked the trail record, it shows:
Version 18.1.0.0.0 OGGCORE_18.1.0.0.0_PLATFORMS_180928.0432
Copyright (C) 1995, 2018, Oracle and/or its affiliates. All rights reserved.
Logdump 822 >open /ogg/ora12c/dirdat/aa000014157
Current LogTrail is /ogg/ora12c/dirdat/aa000014157
Logdump 823 >ggstoken details on
Logdump 824 >ghdr on
Logdump 825 >pos 196854618
Reading forward from RBA 196854618
Logdump 826 >n
___________________________________________________________________
Hdr-Ind : E (x45) Partition : . (x0c)
UndoFlag : . (x00) BeforeAfter: A (x41)
RecLength : 247 (x00f7) IO Time : 2022/02/22 15:46:40.000.000
IOType : 5 (x05) OrigNode : 255 (xff)
TransInd : . (x03) FormatType : R (x52)
SyskeyLen : 0 (x00) Incomplete : . (x00)
AuditRBA : 23328 AuditPos : 104349508
Continued : N (x00) RecCount : 1 (x01)
2022/02/22 15:46:40.000.000 Insert Len 247 RBA 196854618
Name: CLM_TT.WF_TODOITEM (TDR Index: 9)
After Image: Partition 12 G s
0000 000a 0000 0000 0000 005d f097 0001 000a 0000 | ...........]........
0000 0000 0000 0191 0002 0011 0000 000d 576f 726b | ................Work
2041 7373 6967 6e65 6400 0300 1f00 0032 3032 322d | Assigned......2022-
3032 2d32 323a 3135 3a34 363a 3139 2e30 3030 3030 | 02-22:15:46:19.00000
3030 3030 0004 0005 0000 0001 4900 0500 1f00 0032 | 0000........I......2
3032 322d 3032 2d32 323a 3135 3a34 363a 3139 2e30 | 022-02-22:15:46:19.0
3030 3030 3030 3030 0006 001f ffff 3139 3030 2d30 | 00000000......1900-0
GGS tokens:
TokenID x52 'R' ORAROWID Info x00 Length 20
4141 6f63 744d 4146 2b41 4149 5848 6841 4141 0001 | AAoctMAF+AAIXHhAAA..
TokenID x4c 'L' LOGCSN Info x00 Length 12
3336 3039 3939 3033 3833 3931 | 360999038391
TokenID x36 '6' TRANID Info x00 Length 14
302e 3231 2e36 372e 3636 3239 3430 | 0.21.67.662940
TokenID x69 'i' ORATHREADID Info x01 Length 2
0001 | ..
Logdump 827 >
Kindly advise what could be the possible cause.
Regards,
Roshan