Performance DB vs Java Edition
888383Sep 15 2011 — edited Sep 16 2011Hi there,
we've been investigating wether a (and when which) NoSQL-DB outperforms a standard SQL solution for our usecase:
500.000 records bulk insert
2k - 5k per record
With 2 Indices
Using Java API
Multi OS: Linux, Windows, z/OS
Berkeley DB performed best (bulk insert, selects and deletes) on our Windows-Tests (with 2GB memory).
Unfortunately theres a huge gap between Berkley DB and its Java Edition performance.
For a 20.000 record chunk insert Berkeley DB (using BTREE) takes 2000 - 3500 ms. When the cache is full and the first cache misses appear the inserting time grows and stays at arround 8000 ms. After a while st_cacheMiss decreases again resulting in 2000 - 3500 ms per insert.
The JE starts with 4500 - 5000 ms per insert. After the first 200.000 records the cache is full and the nCacheMiss grows rapidly along with the inserting time (11s, 13s, 16s, ... 30s and even longer). There is no constancy like the non Java version shows to have.
Is there any way to reach the Berkeley DB's performance with the JE? Esp. the constant insert time.
We've tried already different CacheModes but default mode is the best.
Setting je.log.faultReadSize=5120 leads to a slighter better perfomance, but still the inserting gets slower and slower.
Since we need z/OS support we'd prefer to use the JE.
Thanks for any help.
Jonas