Skip to Main Content

Intelligent Advisor

Announcement

For appeals, questions and feedback about Oracle Forums, please email oracle-forums-moderators_us@oracle.com. Please ask technical questions in the appropriate category. Thank you!

OPA Performance Questions - Do unused entities impact performance? + Is the CSV input to the batch processor known to be fast

IsamuMay 9 2023

Hi OPA experts,

Can someone help provide guidance for my OPA Batch Processor performance related questions? I'm looking for ways to improve the performance aside from the stuff mentioned in the help file.

Scenario:

Complex multi-use rulebase used for different contexts. Some contexts require the use of multiple entities. Each entity has many inferred relationships. Whenever these entities are being used, the performance doesn't matter.

However, there is one context where the performance matters--and in this contextm we're using the Batch Processor. For this batch processing context, we're only using 1 entity (in addition to global) and therefore, not using/not caring about the other complex entities with many inferred relationships. For these unused entities, we're mapping them in as 'containment complete' via mapping in empty placeholder DB tables.

Sample Data Model
Global
-child entity A w/ some inferred relationships
-child entity B w/ many inferred relationships
-child entity C w/ many inferred realtionships
-child entity D w/ many inferred realtionships
-child entity E w/ many inferred realtionships

Sample Input Data for Batch Processing
Global
- instance 1 of child entity A
- instance 2 of child entity A
- 'Containment Complete' for child entities: B, C, D, E --facilitated with mapping in empty placeholder DB tables

Questions:

  1. For this Batch Processing context, would the fact that many inferred relationship rules exist for child entities B, C, D, E affect the performance despite these entities not being used for the determination? i.e. If I were to rip-out and re-write a rulebase only for the batch processing context, would I get a significant performance increase (i.e. >4x faster)?

  2. Data is currently mapped in from DB tables. Will processing data via CSVs faster than processing data via the DB? Is there a max file size to CSVs? For our scenario, we estimate 2 million global records and 5 million records for child entity A.

An alternative I'm thinking about is to generate CSVs and have OPA process the CSVs, then write back to the DB rather than using the jdbc driver to interact directly with the DB.

  1. Does sorting the Foreign Key data in the child tables improve performance for DB/CSV Processing?
This post has been answered by Richard Napier on May 11 2023
Jump to Answer
Comments
Post Details
Added on May 9 2023
3 comments
373 views