Hey Team,
We are seeing
Found one Java-level deadlock:
=============================
"IOHub#1: Worker[channel:java.nio.channels.SocketChannel[connected local=/10.89.0.2:50000 remote=100.105.134.190/100.105.134.190:37486]] / Computer.threadPoolForRemoting [#9570]":
waiting to lock monitor 0x00007f7ca0009500 (object 0x00007f9c3e6fb930, a org.jenkinsci.remoting.protocol.impl.ConnectionHeadersFilterLayer),
which is held by "TCP agent connection handler #52086 with /100.105.134.190:37486"
"TCP agent connection handler #52086 with /100.105.134.190:37486":
waiting for ownable synchronizer 0x00007f9c3e6f5cb0, (a java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync),
which is held by "IOHub#1: Worker[channel:java.nio.channels.SocketChannel[connected local=/10.89.0.2:50000 remote=100.105.134.190/100.105.134.190:37486]] / Computer.threadPoolForRemoting [#9570]"
Java stack information for the threads listed above:
===================================================
"IOHub#1: Worker[channel:java.nio.channels.SocketChannel[connected local=/10.89.0.2:50000 remote=100.105.134.190/100.105.134.190:37486]] / Computer.threadPoolForRemoting [#9570]":
at org.jenkinsci.remoting.protocol.FilterLayer.onRecvRemoved(FilterLayer.java:134)
- waiting to lock <0x00007f9c3e6fb930> (a org.jenkinsci.remoting.protocol.impl.ConnectionHeadersFilterLayer)
at org.jenkinsci.remoting.protocol.ProtocolStack$Ptr.getNextRecv(ProtocolStack.java:922)
at org.jenkinsci.remoting.protocol.ProtocolStack$Ptr.onRecv(ProtocolStack.java:671)
at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.processRead(SSLEngineFilterLayer.java:367)
at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.onRecv(SSLEngineFilterLayer.java:119)
at org.jenkinsci.remoting.protocol.ProtocolStack$Ptr.onRecv(ProtocolStack.java:677)
at org.jenkinsci.remoting.protocol.NetworkLayer.onRead(NetworkLayer.java:137)
at org.jenkinsci.remoting.protocol.impl.NIONetworkLayer.ready(NIONetworkLayer.java:161)
at org.jenkinsci.remoting.protocol.IOHub$OnReady.run(IOHub.java:789)
at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)
at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:68)
at jenkins.util.ErrorLoggingExecutorService.lambda$wrap$0(ErrorLoggingExecutorService.java:51)
at jenkins.util.ErrorLoggingExecutorService$$Lambda$737/0x00007f7a11779840.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@11.0.26-ea/ThreadPoolExecutor.java:1128)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@11.0.26-ea/ThreadPoolExecutor.java:628)
at java.lang.Thread.run(java.base@11.0.26-ea/Thread.java:834)
"TCP agent connection handler #52086 with /100.105.134.190:37486":
at jdk.internal.misc.Unsafe.park(java.base@11.0.26-ea/Native Method)
- parking to wait for <0x00007f9c3e6f5cb0> (a java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync)
at java.util.concurrent.locks.LockSupport.park(java.base@11.0.26-ea/LockSupport.java:194)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(java.base@11.0.26-ea/AbstractQueuedSynchronizer.java:885)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireShared(java.base@11.0.26-ea/AbstractQueuedSynchronizer.java:1009)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireShared(java.base@11.0.26-ea/AbstractQueuedSynchronizer.java:1324)
at java.util.concurrent.locks.ReentrantReadWriteLock$ReadLock.lock(java.base@11.0.26-ea/ReentrantReadWriteLock.java:738)
at org.jenkinsci.remoting.protocol.ProtocolStack$Ptr.getNextSend(ProtocolStack.java:850)
at org.jenkinsci.remoting.protocol.ProtocolStack$Ptr.doSend(ProtocolStack.java:694)
at org.jenkinsci.remoting.protocol.impl.ConnectionHeadersFilterLayer.doSend(ConnectionHeadersFilterLayer.java:499)
- locked <0x00007f9c3e6fb930> (a org.jenkinsci.remoting.protocol.impl.ConnectionHeadersFilterLayer)
at org.jenkinsci.remoting.protocol.ProtocolStack$Ptr.doSend(ProtocolStack.java:700)
at org.jenkinsci.remoting.protocol.ApplicationLayer.write(ApplicationLayer.java:156)
at org.jenkinsci.remoting.protocol.impl.ChannelApplicationLayer.start(ChannelApplicationLayer.java:259)
at org.jenkinsci.remoting.protocol.ProtocolStack.init(ProtocolStack.java:209)
at org.jenkinsci.remoting.protocol.ProtocolStack$Builder.build(ProtocolStack.java:563)
at org.jenkinsci.remoting.engine.JnlpProtocol4Handler.handle(JnlpProtocol4Handler.java:156)
at jenkins.slaves.JnlpSlaveAgentProtocol4.handle(JnlpSlaveAgentProtocol4.java:177)
at hudson.TcpSlaveAgentListener$ConnectionHandler.run(TcpSlaveAgentListener.java:276)
Found 1 deadlock.
I did read some issues reported for ReentrantReadWriteLock . Is there a way to understand why is this thread BLOCKED on a thread which isn't active. Is there any version for JDK 11 with this patch?