load average of over 800 000??? Sluggish box but CPU is unused?
807557Apr 21 2009 — edited Apr 23 2009Hi,
I have a X4600 running Solaris 10u6 with 32G ram and 4 4core cpu's. When i boot the server and start Glassfish, top shows the following:
load averages: 838860 838860 8388608.0; up 0+00:11:11 17:51:24
44 processes: 43 sleeping, 1 on cpu
CPU states: 99.6% idle, 0.2% user, 0.1% kernel, 0.0% iowait, 0.0% swap
Kernel: 461 ctxsw, 17 trap, 760 intr, 465 syscall, 16 flt
Memory: 32G phys mem, 28G free mem, 2048M total swap, 2048M free swap
PID USERNAME LWP PRI NICE SIZE RES STATE TIME CPU COMMAND
1433 root 96 59 0 932M 366M sleep 2:53 5.09% java
1434 root 96 59 0 924M 390M sleep 3:15 5.04% java
1470 root 1 59 0 5980K 1940K sleep 0:01 1.28% sshd
1080 root 1 59 0 1416K 632K sleep 0:03 1.26% utmpd
1432 root 95 59 0 929M 365M sleep 2:12 0.10% java
1435 root 92 59 0 916M 337M sleep 1:54 0.01% java
1480 root 1 59 0 3140K 1852K cpu/12 0:00 0.01% top
1389 root 61 59 0 621M 111M sleep 0:57 0.00% java
1430 noaccess 18 59 0 167M 105M sleep 0:22 0.00% java
1157 root 2 59 0 2796K 1600K sleep 0:00 0.00% automountd
855 root 30 59 0 6164K 2956K sleep 0:00 0.00% nscd
806 root 12 59 0 12M 11M sleep 1:35 0.00% svc.startd
808 root 19 59 0 8012K 6844K sleep 0:38 0.00% svc.configd
1218 root 1 59 0 7384K 1840K sleep 0:00 0.00% sendmail
1051 root 4 59 0 5460K 3600K sleep 0:02 0.00% inetd
866 daemon 3 59 0 4212K 2324K sleep 0:02 0.00% kcfd
1163 root 14 59 0 3764K 1448K sleep 0:02 0.00% syslogd
1081 root 1 59 0 2396K 1244K sleep 0:02 0.00% ttymon
1221 root 1 59 0 9568K 6468K sleep 0:00 0.00% snmpd
1447 root 1 59 0 6232K 3732K sleep 0:00 0.00% sshd
1444 root 1 59 0 6232K 3704K sleep 0:00 0.00% sshd
1268 root 2 59 0 3760K 2076K sleep 0:00 0.00% snmpXdmid
1450 root 1 59 0 5980K 1944K sleep 0:00 0.00% sshd
1452 root 1 59 0 2924K 1820K sleep 0:00 0.00% bash
1472 root 1 59 0 2924K 1820K sleep 0:00 0.00% bash
1267 root 1 59 0 3168K 1624K sleep 0:00 0.00% dmispd
1012 daemon 1 59 0 2764K 1616K sleep 0:00 0.00% statd
1219 smmsp 1 59 0 7512K 1584K sleep 0:00 0.00% sendmail
1234 root 1 59 0 5704K 1560K sleep 0:00 0.00% dtlogin
1207 root 1 59 0 3840K 1452K sleep 0:00 0.00% sshd
Everything looks like the server is idle, except that the load average is sooo high, and when ssh'ing into the box, it is extremely slow. If i hit enter, it takes several seconds to respond and the glassfish server can barely server pages. What on earth could be going on? Syslog shows nothing, dmesg shows nothing and generally I'm stumped!
Any suggestions on where to look???
Thanks so much
Paul