Skip to Main Content

ODP.NET

Announcement

For appeals, questions and feedback, please email oracle-forums_moderators_us@oracle.com

Connection request timed out

user5819206Jan 31 2024

We are getting an OracleException with this message “Connection request timed out” intermittently in our production environment. Our production environment is Kubernetes using an Ubuntu Jammy image on an application using net8.0. We have our application deployed to 6 clusters with 3 PODs per cluster. The application will make anywhere from 10k to 40k DB queries per 5 min.

The problem is that one of the PODs will start having connection problems to the DB that manifest as the above error with this stack trace:

   at OracleInternal.ConnectionPool.PoolManager`3.Get(ConnectionString csWithDiffOrNewPwd, Boolean bGetForApp, OracleConnection connRefForCriteria, String affinityInstanceName, Boolean bForceMatch)
  at OracleInternal.ConnectionPool.OraclePoolManager.Get(ConnectionString csWithNewPassword, Boolean bGetForApp, OracleConnection connRefForCriteria, String affinityInstanceName, Boolean bForceMatch)
  at OracleInternal.ConnectionPool.OracleConnectionDispenser`3.Get(ConnectionString cs, PM conPM, ConnectionString pmCS, SecureString securedPassword, SecureString securedProxyPassword, OracleConnection connRefForCriteria)
  at Oracle.ManagedDataAccess.Client.OracleConnection.Open()\n   at System.Data.Common.DbConnection.OpenAsync(CancellationToken cancellationToken)
--- End of stack trace from previous location ---
  at <redacted>.DatabaseHelper.DatabaseService.GetConnectionAsync(String name)
  at <redacted>.DatabaseHelper.DatabaseService.ExecuteReaderAsync(String target, String statement, OracleParameter[] parameters, Action`1 handler)

This POD will continue to have issues until we restart the container, then it is just fine. During this time all the other PODs are perfectly happy. When we query the DB to see how many sessions are being used by our application with this query:

select substr(machine,0, instr(machine, '\') - 1), count(1)
from v$session
where machine like '<redacted>%' and schemaname = '<redacted>'
group by machine
order by count(1) desc

We can see that the affected machines aren't having high sessions. As an example here is our connection string:

Data Source=(DESCRIPTION=(ADDRESS_LIST=(ADDRESS=(PROTOCOL=TCP)(HOST=<redacted>)(PORT=1522)))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=<redacted>))); Password=<redacted>; User ID=<redacted>; Min Pool Size=1; Max Pool Size=90; Connection Lifetime=300;

Tonight when I got sev'd out because of the issue, while the issue was still going on the above query showed it has 14 sessions. So it should have been able to create a new connection to add to the pool, shouldn't it?

I am at a loss on how to diagnose this issue. Partly because our production environment is very locked down so getting much diagnostic information can be a challenge.

Comments
Post Details
Added on Jan 31 2024
1 comment
47 views