Berkeley DB – performance of inserts into large database.
684858Feb 14 2009 — edited Feb 16 2009Berkeley DB – performance of inserts into large database.
I have read BDB performance benchmarks 
http://www.oracle.com/technology/products/berkeley-db/pdf/berkeley-db-perf.pdf
The speed of single-record insertion should be about several thousand operations per second on commodity hardware 
I’ve tried to test it on my computer
Ubuntu Linux 64, core2quad 2.5Ghz, 8Gb RAM, 4kb cluster, SSD PATRIOT 128Gb 
Berkeley db 4.6
The test was 
To create a database with hash access.
Insert random 64byte key and 1 kbyte data in loop. 
Print an average insertion speed on the screen every 5 seconds.
At the beginning, when the quantity of data was low, the insertion of new rows was fast.
starting, DB_HASH
cache: 0/267072/1 --------- default cache
pagesize: 2048
TimeInterval: 5000000, KEY64, DATA1024
random keys, seed = 12345
281738 56347 / sec
403267 24305 / sec
435074 6361 / sec
461809 5347 / sec
491495 5937 / sec
Once the quantity of data had reached 3 million keys, the insertion became slow.
The more data is inserted into the database, the slower it works. 
3248817 179 / sec
3253905 1017 / sec
3257165 652 / sec
3258585 284 / sec
d
3262057 694 / sec
53443487 8 / sec !!!
53443524 7 / sec
53443550 5 / sec
I’ve got an interesting dependence – a large cache (about 2-3 gigabyte) makes insertion into 1 million rows database as slow as small default cache into 3 million rows database.
Where is the problem? Double caching (by BDB and by OS)?
What can i do to make the insertion as fast as it was written in Oracle benchmark tests?
Please, tell any opinions, i need your help very much.
Edited by: user5156754 on 14.02.2009 7:11