Skip to Main Content

Berkeley DB Family

Announcement

For appeals, questions and feedback about Oracle Forums, please email oracle-forums-moderators_us@oracle.com. Technical questions should be asked in the appropriate category. Thank you!

Interested in getting your voice heard by members of the Developer Marketing team at Oracle? Check out this post for AppDev or this post for AI focus group information.

[PATCH] The memory is not re-mapped after libdb extends the __db file

MIZUTA TakeshiSep 1 2021 — edited Sep 3 2021

When the __db file size limit is reached, libdb calls lseek() and write() to extend the file.

19 int
20 __db_file_extend(env, fhp, size)
<SNIP>
51 pages = (db_pgno_t)((size - sizeof(buf)) / MEGABYTE);
52 relative = (u_int32_t)((size - sizeof(buf)) % MEGABYTE);
53 if ((ret = __os_seek(env, fhp, pages, MEGABYTE, relative)) == 0)
54 ret = __os_write(env, fhp, &buf, sizeof(buf), &nw);

However, after expanding the __db file, it is not syncing with memory.
mmap has the following notes.

From mmap(2) manpage:

A file is mapped in multiples of the page size. For a file that is not
a multiple of the page size, the remaining memory is zeroed when
mapped, and writes to that region are not written out to the file. The
effect of changing the size of the underlying file of a mapping on the
pages that correspond to added or removed regions of the file is
unspecified.

From mmap in IEEE Std 1003.1-2017:

If the size of the mapped file changes after the call to mmap() as a result of
some other operation on the mapped file, the effect of references to portions
of the mapped region that correspond to added or removed portions of the file
is unspecified.

Therefore, one of the following deals are required.

  • Call munmap()/mmap() to synchronize memory during lseek() and write()
  • Get the maximum size __db file in advance instead of gradually expanding the __db file.

The following is the latter patch:

--------
diff --git a/db-18.1.40/src/os/os_map.c b/db-18.1.40/src/os/os_map.c
index dcf2c23..83a79a8 100644
--- a/db-18.1.40/src/os/os_map.c
+++ b/db-18.1.40/src/os/os_map.c
@@ -231,15 +231,7 @@ __os_attach(env, infop, rp)
if (rp->max < rp->size)
rp->max = rp->size;
if (ret == 0 && F_ISSET(infop, REGION_CREATE)) {
-#ifdef HAVE_MLOCK
- /*
- * When locking the region in memory extend it fully so that it
- * can all be mlock()'d now, and not later when paging could
- * interfere with the application. [#21379]
- */
- if (F_ISSET(env, ENV_LOCKDOWN))
- rp->size = rp->max;
-#endif

  •    rp->size = rp->max;  
      if (F\_ISSET(dbenv, DB\_ENV\_REGION\_INIT))  
          ret = \_\_db\_file\_write(env, infop->fhp,  
            rp->size / MEGABYTE, rp->size % MEGABYTE, 0x00);  
    

--------

Here's the patch:
https://github.com/miztake/db-18.1.40/commit/45446f3857e6a972a3477ac01b8fb578fce03056

Do you have any opinions about this patch?

Comments

MartinBach-Oracle Jan 8 2025

Hi Salomon,

please have a look at this blog post written by @ulrike-schwinn-oracle :

https://blogs.oracle.com/coretec/post/easy-sql-statement-tracking-in23c

I hope this answers your question, if not, please shout!

- Martin

Solomon Yakobson Jan 8 2025

@martinbach-oracle - No, it doesn't answer my question. Article you pointed to shows uses:

SQL> alter system set sql_history_enabled=true scope=both;

And in my post I said “Works fine when enabled on system level”. My question was about

SQL> ALTER SESSION SET SQL_HISTORY_ENABLED = TRUE;

where I showed SQL history was NOT captured even though it should be based on SQL_HISTORY_ENABLED:

Modifiable **ALTER SESSION**, ALTER SYSTEM

SY.

MartinBach-Oracle Jan 8 2025

As per the article I shared the situation is as follows at the moment

  • You must enable SQL history PDB-wide (only a DBA can do that) so there's a certain level of control over the feature
  • Your session has access to the SQL history
  • If you don't want to record anything, set sql_history_enabled to false.

I'm currently assessing if that's intended behaviour (in which case the documentation should be amended) or a feature not working as it should (in which case it needs fixing). The parameter is indeed session-modifyable, but not in the sense you expected.

We'll keep you posted.

- Martin

Solomon Yakobson Jan 8 2025

Do you mean it must be enabled on system level and not on session level and all session can do is disabe it for the session?

SY.

MartinBach-Oracle Jan 14 2025

Yes,

that's correct as of Oracle Database Free 23.6.

- Martin

Solomon Yakobson Jan 14 2025

Thanks Martin, I hope this will be added to 23AI docs soon.

SY.

1 - 6

Post Details

Added on Sep 1 2021
1 comment
367 views