Berkeley DB Family

We have been using je 3.2.21 for over a year. For the past several months, we experienced sporadic cleaner thread blocking issue. When that happened, the application threads writing to database stopped processing because the last transaction was never finished. This can last for more than 2 hours and we had to bounce the application to rectify the situation. Thread dump shows that cleaner thread was holding locks and writing threads were waiting for those locks.

Here is some related information. Our total data file size ranges from 5GB to 30GB with 50% utilization. The cache size is about 800MB. There seems to be no correlation of occurrence frequency to dataset size.

I searched in this forum for cleaner thread known issues and did not find why this is happening.

I have a couple of questions regarding this. Hope experts on this forum can help.
1) Is this a known bug that has been fixed by later releases of je?
2) Is it by design that when cleaner threading is doing its job, other writing threads are blocked?
3) What parameters we can tweak to avoid this blocking?

Following are from thread dump

"Cleaner-1" daemon prio=1 tid=0x0000002e2b36c7b0 nid=0x7dd0 runnable [0x0000000045787000..0x0000000045787b30]
at java.io.UnixFileSystem.getBooleanAttributes0(Native Method)
at java.io.UnixFileSystem.getBooleanAttributes(Unknown Source)
at java.io.File.exists(Unknown Source)
at com.sleepycat.je.cleaner.UtilizationProfile.putFileSummary(UtilizationProfile.java:720)
- locked <0x0000002aa19ba168> (a com.sleepycat.je.cleaner.UtilizationProfile)
at com.sleepycat.je.cleaner.UtilizationProfile.flushFileSummary(UtilizationProfile.java:680)
at com.sleepycat.je.cleaner.UtilizationTracker.evictMemory(UtilizationTracker.java:118)
at com.sleepycat.je.evictor.Evictor.evictBatch(Evictor.java:313)
at com.sleepycat.je.evictor.Evictor.doEvict(Evictor.java:253)
- locked <0x0000002aa19c44e0> (a com.sleepycat.je.evictor.Evictor)
at com.sleepycat.je.evictor.Evictor.doCriticalEviction(Evictor.java:278)
at com.sleepycat.je.cleaner.FileProcessor.processFile(FileProcessor.java:474)
at com.sleepycat.je.cleaner.FileProcessor.doClean(FileProcessor.java:257)
- locked <0x0000002aa19b9f08> (a com.sleepycat.je.cleaner.FileProcessor)
at com.sleepycat.je.cleaner.FileProcessor.onWakeup(FileProcessor.java:160)
at com.sleepycat.je.utilint.DaemonThread.run(DaemonThread.java:191)
at java.lang.Thread.run(Unknown Source)

"INCompressor" daemon prio=1 tid=0x0000002e2b36d200 nid=0x7dcf waiting for monitor entry [0x0000000045686000..0x0000000045686bb0]
at com.sleepycat.je.evictor.Evictor.doEvict(Evictor.java:239)
- waiting to lock <0x0000002aa19c44e0> (a com.sleepycat.je.evictor.Evictor)
at com.sleepycat.je.evictor.Evictor.doCriticalEviction(Evictor.java:278)
at com.sleepycat.je.incomp.INCompressor.findDBAndBIN(INCompressor.java:803)
at com.sleepycat.je.incomp.INCompressor.doCompress(INCompressor.java:403)
- locked <0x0000002aa19c45c0> (a com.sleepycat.je.incomp.INCompressor)
at com.sleepycat.je.incomp.INCompressor.onWakeup(INCompressor.java:342)
- locked <0x0000002aa19c45c0> (a com.sleepycat.je.incomp.INCompressor)
at com.sleepycat.je.utilint.DaemonThread.run(DaemonThread.java:191)
at java.lang.Thread.run(Unknown Source)

"POOLED-INFOBUS-2" prio=1 tid=0x0000002e2a7ffac0 nid=0x7bec waiting for monitor entry [0x000000004166a000..0x000000004166ac30]
at com.sleepycat.je.evictor.Evictor.doEvict(Evictor.java:239)
- waiting to lock <0x0000002aa19c44e0> (a com.sleepycat.je.evictor.Evictor)
at com.sleepycat.je.evictor.Evictor.doCriticalEviction(Evictor.java:278)
at com.sleepycat.je.dbi.CursorImpl.cloneCursor(CursorImpl.java:330)
at com.sleepycat.je.Cursor.putAllowPhantoms(Cursor.java:854)
at com.sleepycat.je.Cursor.putNoNotify(Cursor.java:798)
at com.sleepycat.je.Cursor.putInternal(Cursor.java:752)
at com.sleepycat.je.Database.putInternal(Database.java:662)
at com.sleepycat.je.Database.put(Database.java:608)
at com.ml.pstt.core.bdb.BerkeleyDb$2.doWork(BerkeleyDb.java:161)
at com.sleepycat.collections.TransactionRunner.run(TransactionRunner.java:229)
at com.ml.pstt.core.bdb.BerkeleyDb.saveToDb(BerkeleyDb.java:178)
at com.ml.pstt.core.bdb.BerkeleyDb.saveToDb(BerkeleyDb.java:195)

"POOLED-INFOBUS-1" prio=1 tid=0x0000002e2a7ff310 nid=0x7beb waiting for monitor entry [0x0000000041569000..0x0000000041569cb0]
at com.sleepycat.je.evictor.Evictor.doEvict(Evictor.java:239)
- waiting to lock <0x0000002aa19c44e0> (a com.sleepycat.je.evictor.Evictor)
at com.sleepycat.je.evictor.Evictor.doCriticalEviction(Evictor.java:278)
at com.sleepycat.je.dbi.CursorImpl.cloneCursor(CursorImpl.java:330)
at com.sleepycat.je.Cursor.putAllowPhantoms(Cursor.java:854)
at com.sleepycat.je.Cursor.putNoNotify(Cursor.java:798)
at com.sleepycat.je.Cursor.putInternal(Cursor.java:752)
at com.sleepycat.je.Database.putInternal(Database.java:662)
at com.sleepycat.je.Database.put(Database.java:608)
at com.ml.pstt.core.bdb.BerkeleyDb$2.doWork(BerkeleyDb.java:161)
at com.sleepycat.collections.TransactionRunner.run(TransactionRunner.java:229)
at com.ml.pstt.core.bdb.BerkeleyDb.saveToDb(BerkeleyDb.java:178)
at com.ml.pstt.core.bdb.BerkeleyDb.saveToDb(BerkeleyDb.java:195)



Thanks