Performance problems - application running very slow with many connections
618026Dec 16 2009 — edited Dec 22 2009Hi!
I'm are having a problem with an application running very slow in production. In test env. the application is running fine.
I thought the problem would be solved when adding two indexes to a FK in a referenced table (table CART has an id which is a FK in CART_ITEM).
The thing with this is that it did solve the problems in test but did almost nothing in production.....
I have compared the settings in the both environments and as of now the following things has been verified:
* JVM - all JVM settings are identical
* Index - all indexes in the database are the same
* Logging - less logs are printed in production (we only log errors in production)
* Hardware - Both production and test is unning on identical hardware
* Database fills - I have filled the test database with production like numbers of records
Still, running load tests in the test env. results in no problem at all now (before indexing we could sink the application) but users complain that it's very, very slow in production.
I have taken a look at the weblogic console monitoring tools and found some interesting differences and wonder if you have any idea why the numbers are so different in test and production?
date ExecuteThreadTotalCount PendingUserRequestCount StandbyThreadCount HoggingThreadCount QueueLength ActiveConnectionsCurrentCount WaitingForConnectionHighCount ActiveConnectionsAverageCount WaitingForConnectionCurrentCount State CurrCapacity LeakedConnectionCount ConnectionDelayTime ActiveConnectionsHighCount CurrCapacityHighCount ConnectionsTotalCount NumAvailable OpenSessionsCurrentCount
MONITOR FROM TEST (the application has no load at night)
00:03:49 54 0 2 0 0 4 0 4 0 Running 4 0 156 15 14 95 0 0
MONITOR FROM PRODUCTION (the application has no load at night)
00:02:40 65 0 47 16 0 22 26 22 0 Running 22 0 129 40 39 640 0 9
MONITOR FROM TEST (the application has maximum load)
10:39:03 54 0 2 1 0 5 0 4 0 Running 14 0 167 15 14 123 9 365
MONITOR FROM PRODUCTION (the application has maximum load)
11:47:58 65 0 20 25 0 34 26 28 0 Running 36 0 132 40 39 763 2 169
As you can see, there are big differences in some cases, especially these rows:
StandbyThreadCount
HoggingThreadCount
ActiveConnectionsCurrentCount
ActiveConnectionsAverageCount
CurrCapacity
ActiveConnectionsHighCount
CurrCapacityHighCount
ConnectionsTotalCount
The thing I'm mostly interested in is that these values are high even at night, when no one uses the application?
I realise that you probably will need at lot more information to be able to draw any conclusions on what might be stalling the production environment but tell me what you need (an maybe how I shall get it =)) and I'll try to provide it!
Thank you!
/Paul