Skip to Main Content

Berkeley DB Family

Announcement

For appeals, questions and feedback about Oracle Forums, please email oracle-forums-moderators_us@oracle.com. Technical questions should be asked in the appropriate category. Thank you!

Interested in getting your voice heard by members of the Developer Marketing team at Oracle? Check out this post for AppDev or this post for AI focus group information.

Over 4GB read-only mmap B-tree fails to find records

User_DAIKJJun 23 2021

Systems: Windows 10, RHLE, HP-UX, always a 64-bit compile.
BDB: 5.x, 6.x, 18.x
This problem exists on every system I have tried, with every version of BDB I can get my hands on.
Create a b-tree with a size just over 4GB. I use a 64-bit unsigned key and a 24-byte record, which takes about 67,200,300 records to get over 4GB (4,341,264,384). The same key/record setup with 66,200,300 records is just under 4GB and does NOT exhibit the problem.
After generation, reopen the database "read-only" (DB_RDONLY) with an environment, cache, and mmap setting to allow BDB to mmap the file. I use a 5GB cache and set the mmap limit to 16GB.
Start looking up the records. Very quickly db->get will fail with BDB0073 DB_NOTFOUND. The lib also produces a BDB3008 message: "dirty flag set for readonly file page". If you keep calling db->get, then the lib will panic.
Reopen the database, but this time pass the DB_NOMMAP to the db->open call. Notice how all the keys are found.
Generate a database with 66,200,300 records of the same type as above. Notice that the file is just under 4GB (4,247,216,128).
Reopen the under 4GB database the same as step #2, i.e. read-only, cache and mmap set to allow BDB to mmap the file. Notice how all the keys are found.
Use db_stat to see there is nothing wrong found with either db file.
Use db_dump to view the first 100 or so records and notice that the key that failed to be found above, is indeed in the database file.
Using a different OS, compiler, or system has not made any difference with this problem, neither has 4096 vs 8192 size pages, and making sure the page size matches the filesystem.
Something else that is strange. If you generate the database, sync it, close it, then reopen it read-only, all in the same run of a program within the same db environment, then the database over 4GB checks out fine, i.e. all the keys are found. However, exit the program and reopen the database read-only for checking, and the same program / code-path fails as above.
Any insight, questions, or suggestion would be greatly appreciated.

Comments

MartinBach-Oracle Jan 8 2025

Hi Salomon,

please have a look at this blog post written by @ulrike-schwinn-oracle :

https://blogs.oracle.com/coretec/post/easy-sql-statement-tracking-in23c

I hope this answers your question, if not, please shout!

- Martin

Solomon Yakobson Jan 8 2025

@martinbach-oracle - No, it doesn't answer my question. Article you pointed to shows uses:

SQL> alter system set sql_history_enabled=true scope=both;

And in my post I said “Works fine when enabled on system level”. My question was about

SQL> ALTER SESSION SET SQL_HISTORY_ENABLED = TRUE;

where I showed SQL history was NOT captured even though it should be based on SQL_HISTORY_ENABLED:

Modifiable **ALTER SESSION**, ALTER SYSTEM

SY.

MartinBach-Oracle Jan 8 2025

As per the article I shared the situation is as follows at the moment

  • You must enable SQL history PDB-wide (only a DBA can do that) so there's a certain level of control over the feature
  • Your session has access to the SQL history
  • If you don't want to record anything, set sql_history_enabled to false.

I'm currently assessing if that's intended behaviour (in which case the documentation should be amended) or a feature not working as it should (in which case it needs fixing). The parameter is indeed session-modifyable, but not in the sense you expected.

We'll keep you posted.

- Martin

Solomon Yakobson Jan 8 2025

Do you mean it must be enabled on system level and not on session level and all session can do is disabe it for the session?

SY.

MartinBach-Oracle Jan 14 2025

Yes,

that's correct as of Oracle Database Free 23.6.

- Martin

Solomon Yakobson Jan 14 2025

Thanks Martin, I hope this will be added to 23AI docs soon.

SY.

1 - 6

Post Details

Added on Jun 23 2021
1 comment
269 views