Hi everyone, I’m a newcomer to Oracle TimesTen ScaleOut, working at a securities/financial trading company. I’d like to seek advice on an optimal database design solution for our hybrid architecture.
Current Setup We use:
- TimesTen ScaleOut → OLTP (low-latency, high-throughput trading operations)
- PostgreSQL → OLAP (analytics, reporting)
- Current Sync Approach & Its Problems Since TimesTen ScaleOut does not natively support CDC (Change Data Capture), our current sync design is:
- A service layer interacts with TimesTen → publishes change messages to Kafka
- A consumer service reads from Kafka → writes/persists to PostgreSQL
This approach introduces two key problems:
- Data reconciliation overhead between the two clusters — handling message loss, ordering, duplicates, and consistency gaps is non-trivial, especially under high trading volume.
- Reconciliation increases read load on the OLTP cluster — the reconciliation process itself requires reading back data from TimesTen to verify consistency, which adds unwanted pressure on a latency-sensitive system.
What I’m Looking For Are there alternative architectural approaches to synchronizing TimesTen ScaleOut → PostgreSQL (or achieving OLTP/OLAP separation) that avoid or reduce these problems? Any advice on trade-offs, real-world experience, or references to similar architectures in high-frequency / financial systems would be greatly appreciated. Thanks in advance!