Intermittent LiteSpeed issues on a VPS

M

MikeDVB

Well-Known Member
#1
#1
Ok, so I have a client with a VPS that is running LiteSpeed Enterprise and he usually has around 120~180 active users on his site at any given time.

The server runs great for a day or two and then suddenly it starts taking 15~20 seconds for the web server even to respond to requests. When I check the real time status it's processing around 40~50 requests per second with only 50~60 active connections (out of 500).

Load is low - around 1.5~2.5 or so on an 8-Core VPS and memory usage is low as well (around 350mb out of 1.5GB ram). There's no I/O wait and the node itself isn't under any major load or experiencing any I/O wait.

After trying everything from optimizing MySQL, verifying the networking is working right (tested inbound and outbound) and everything else I can think ... even restarting LiteSpeed the issue would not go away.

I went into the WHM Plugin for LiteSpeed and switched back to Apache and everything loads quickly and efficiently as it should with no issues.

Any suggestions?

Memory:
Code: 
09:40:01 AM kbmemfree kbmemused  %memused kbbuffers  kbcached kbswpfree kbswpused  %swpused  kbswpcad
09:50:02 AM   1213304    359560     22.86         0         0         0         0      0.00         0
10:00:01 AM   1194576    378288     24.05         0         0         0         0      0.00         0
10:10:02 AM   1197636    375228     23.86         0         0         0         0      0.00         0
10:20:02 AM   1194812    378052     24.04         0         0         0         0      0.00         0
10:30:01 AM   1192312    380552     24.19         0         0         0         0      0.00         0
10:40:01 AM   1193336    379528     24.13         0         0         0         0      0.00         0
10:50:01 AM   1164544    408320     25.96         0         0         0         0      0.00         0
11:00:01 AM   1174376    398488     25.34         0         0         0         0      0.00         0
11:10:01 AM   1180220    392644     24.96         0         0         0         0      0.00         0
11:20:01 AM   1180568    392296     24.94         0         0         0         0      0.00         0
11:30:01 AM   1181280    391584     24.90         0         0         0         0      0.00         0
11:40:01 AM   1205044    367820     23.39         0         0         0         0      0.00         0
11:50:01 AM   1233140    339724     21.60         0         0         0         0      0.00         0
12:00:01 PM   1214100    358764     22.81         0         0         0         0      0.00         0
12:10:02 PM   1180668    392196     24.94         0         0         0         0      0.00         0
12:20:01 PM   1202596    370268     23.54         0         0         0         0      0.00         0
12:30:01 PM   1219908    352956     22.44         0         0         0         0      0.00         0
12:40:01 PM   1193916    378948     24.09         0         0         0         0      0.00         0
12:50:01 PM   1187904    384960     24.48         0         0         0         0      0.00         0
01:00:01 PM   1198976    373888     23.77         0         0         0         0      0.00         0
01:10:02 PM   1181492    391372     24.88         0         0         0         0      0.00         0
01:20:01 PM   1120080    452784     28.79         0         0         0         0      0.00         0
01:30:02 PM   1172088    400776     25.48         0         0         0         0      0.00         0
01:40:02 PM   1168236    404628     25.73         0         0         0         0      0.00         0
01:50:01 PM   1138932    433932     27.59         0         0         0         0      0.00         0
02:00:01 PM   1197164    375700     23.89         0         0         0         0      0.00         0
02:10:03 PM   1178132    394732     25.10         0         0         0         0      0.00         0
02:20:02 PM   1170332    402532     25.59         0         0         0         0      0.00         0
02:30:01 PM   1186592    386272     24.56         0         0         0         0      0.00         0
02:40:01 PM   1171780    401084     25.50         0         0         0         0      0.00         0
02:50:01 PM   1172592    400272     25.45         0         0         0         0      0.00         0
03:00:01 PM   1212272    360592     22.93         0         0         0         0      0.00         0
03:10:02 PM   1187624    385240     24.49         0         0         0         0      0.00         0
03:20:01 PM   1208592    364272     23.16         0         0         0         0      0.00         0
03:30:01 PM   1220352    352512     22.41         0         0         0         0      0.00         0
03:40:01 PM   1170556    402308     25.58         0         0         0         0      0.00         0
03:50:01 PM   1210896    361968     23.01         0         0         0         0      0.00         0
04:00:01 PM   1192964    379900     24.15         0         0         0         0      0.00         0
04:10:01 PM   1177072    395792     25.16         0         0         0         0      0.00         0
04:20:01 PM   1127396    445468     28.32         0         0         0         0      0.00         0
Average:      1190303    382561     24.32         0         0         0         0      0.00         0
Load:
Code: 
09:40:01 AM   runq-sz  plist-sz   ldavg-1   ldavg-5  ldavg-15
09:50:02 AM         5        84      1.53      1.97      2.01
10:00:01 AM         8        86      2.43      2.10      2.04
10:10:02 AM         1        83      2.14      2.32      2.12
10:20:02 AM         1        86      1.92      1.97      2.04
10:30:01 AM         8        80      2.43      2.29      2.19
10:40:01 AM         5        87      1.41      1.85      2.03
10:50:01 AM         2        88      1.72      1.83      1.92
11:00:01 AM         7        85      2.10      2.05      2.02
11:10:01 AM         8        87      2.63      2.41      2.23
11:20:01 AM         5        89      1.50      2.04      2.14
11:30:01 AM         5        89      1.70      1.88      2.04
11:40:01 AM        13        83      1.73      2.09      2.12
11:50:01 AM         7        79      2.21      2.28      2.25
12:00:01 PM         7        87      1.18      1.80      2.04
12:10:02 PM         4        91      2.49      2.07      2.08
12:20:01 PM         6        87      1.66      1.90      1.99
12:30:01 PM         5        85      1.70      2.00      2.00
12:40:01 PM         4        87      1.40      1.76      1.95
12:50:01 PM         5        87      1.50      1.64      1.79
01:00:01 PM         7        86      2.56      1.99      1.87
01:10:02 PM        10        88      1.95      2.40      2.19
01:20:01 PM         4        86      1.77      2.19      2.20
01:30:02 PM         9        88      2.03      2.00      2.11
01:40:02 PM         7        90      1.47      1.95      2.08
01:50:01 PM         5        83      1.76      2.02      2.09
02:00:01 PM         5        84      2.26      2.05      2.08
02:10:03 PM        13        90      2.26      2.05      2.05
02:20:02 PM        11        87      1.66      1.91      2.02
02:30:01 PM        12        81      1.73      2.04      2.13
02:40:01 PM         5        89      0.99      1.88      2.06
02:50:01 PM         3        89      0.23      0.48      1.18
03:00:01 PM         8        82      1.46      1.39      1.29
03:10:02 PM         7        87      1.63      2.08      1.76
03:20:01 PM         8        82      1.80      2.08      1.98
03:30:01 PM         5        82      1.68      1.96      2.04
03:40:01 PM         4        88      2.50      2.66      2.38
03:50:01 PM         6        82      2.07      2.09      2.20
04:00:01 PM         8        84      1.96      1.69      1.89
04:10:01 PM         1        89      0.42      0.88      1.39
04:20:01 PM         3       141      1.24      0.68      0.99
Average:            6        85      2.01      2.07      2.09
 
Last edited:
M

MikeDVB

Well-Known Member
#2
#2
CPU:
Code: 
09:40:01 AM       CPU     %user     %nice   %system   %iowait    %steal     %idle
09:50:02 AM       all      9.75      0.09      6.85      0.66      0.00     82.66
10:00:01 AM       all      9.40      0.09      6.64      1.05      0.00     82.81
10:10:02 AM       all      9.42      0.13      6.96      2.46      0.00     81.03
10:20:02 AM       all     10.09      0.09      7.00      0.96      0.00     81.85
10:30:01 AM       all     10.27      0.09      8.46      1.24      0.00     79.94
10:40:01 AM       all      9.19      0.13      6.99      0.70      0.00     82.99
10:50:01 AM       all      9.08      0.09      6.87      0.89      0.00     83.07
11:00:01 AM       all      9.23      0.10      7.09      1.40      0.00     82.18
11:10:01 AM       all      9.13      0.28      7.00      2.23      0.00     81.36
11:20:01 AM       all      8.82      0.09      6.90      1.65      0.00     82.54
11:30:01 AM       all      8.79      0.09      6.69      0.94      0.00     83.49
11:40:01 AM       all      8.79      0.16      6.08      1.74      0.00     83.23
11:50:01 AM       all     10.66      0.10      8.80      1.60      0.00     78.84
12:00:01 PM       all      9.30      0.10      6.86      1.14      0.00     82.60
12:10:02 PM       all      8.75      0.13      6.42      2.49      0.00     82.21
12:20:01 PM       all      9.09      0.10      7.28      0.68      0.00     82.86
12:30:01 PM       all     10.49      0.09      8.45      1.28      0.00     79.68
12:40:01 PM       all      9.63      0.14      6.81      0.86      0.00     82.56
12:50:01 PM       all      9.07      0.09      5.75      0.77      0.00     84.33
01:00:01 PM       all      9.87      0.09      6.24      1.38      0.00     82.42
01:10:02 PM       all      9.29      0.28      5.65      3.54      0.00     81.24
01:20:01 PM       all     10.00      0.09      6.05      1.28      0.00     82.57
01:30:02 PM       all     11.28      0.09      7.18      0.76      0.00     80.69
01:40:02 PM       all      9.76      0.18      6.27      1.02      0.00     82.76
01:50:01 PM       all     10.64      0.09      6.24      1.01      0.00     82.02
02:00:01 PM       all     10.20      0.09      6.31      1.36      0.00     82.03
02:10:03 PM       all     10.20      0.13      6.44      1.51      0.00     81.73
02:20:02 PM       all      8.72      0.09      5.32      1.97      0.00     83.90
02:30:01 PM       all     11.29      0.10      7.38      1.82      0.00     79.41
02:40:01 PM       all      9.65      0.13      5.93      2.46      0.00     81.83
02:50:01 PM       all      1.85      0.10      0.90      0.19      0.00     96.95
03:00:01 PM       all      7.30      0.10      4.75      0.81      0.00     87.04
03:10:02 PM       all      9.39      0.28      6.33      2.84      0.00     81.17
03:20:01 PM       all     10.75      0.09      6.91      1.32      0.00     80.93
03:30:01 PM       all      9.79      0.10      6.02      2.16      0.00     81.93
03:40:01 PM       all      9.85      0.18      6.51      1.08      0.00     82.38
03:50:01 PM       all     10.32      0.09      6.57      1.66      0.00     81.36
04:00:01 PM       all      9.43      0.09      5.79      0.80      0.00     83.89
04:10:01 PM       all      3.57      0.13      1.92      2.20      0.00     92.18
04:20:01 PM       all      2.01      0.09      1.00      0.53      0.00     96.37
Average:          all      9.25      0.18      6.57      1.99      0.00     82.02
Basic LiteSpeed Status:


Networking:
Incoming (From an external server to the VPS)
Code: 
root@server1 [/]# wget http://redacted/test100.zip
--2010-05-03 16:17:16-- http://redacted/test100.zip
HTTP request sent, awaiting response... 200 OK
Length: 104874307 (100M) [application/zip]
Saving to: `test100.zip'

100%[======================================>] 104,874,307 11.8M/s in 9.6s

2010-05-03 16:17:26 (10.5 MB/s) - `test100.zip' saved [104874307/104874307]
Outgoing (From the VPS to an external server)
Code: 
root@server [~]# wget http://redacted/test100.zip
--2010-05-03 16:18:05-- http://redacted/test100.zip
HTTP request sent, awaiting response... 200 OK
Length: 104874307 (100M) [application/zip]
Saving to: `test100.zip'

100%[======================================>] 104,874,307 11.4M/s in 9.7s

2010-05-03 16:18:08 (10.3 MB/s) - `test100.zip' saved [104874307/104874307]
MySQL:
Code: 
[OK] Slow queries: 0% (414/1B)
[OK] Highest usage of available connections: 22% (22/100)
[OK] Key buffer size / total MyISAM indexes: 64.0M/60.8M
[OK] Key buffer hit rate: 99.8% (58M cached / 100K reads)
[OK] Query cache efficiency: 98.2% (366M cached / 373M selects)
[OK] Sorts requiring temporary tables: 9% (137K temp sorts / 1M sorts)
[OK] Temporary tables created on disk: 9% (384K on disk / 3M total)
[OK] Thread cache hit rate: 99% (22 created / 377M connections)
[OK] Table cache hit rate: 31% (679 open / 2K opened)
[OK] Open file limit used: 12% (1K/8K)
[OK] Table locks acquired immediately: 99% (9M immediate / 9M locks)
I'm really at a loss - swapping out LiteSpeed caused everything to speed up. Ping and traceroute also looked golden both forward and reverse paths.

Perhaps the LSAPI/Connector needs tweaking? I checked the LiteSpeed error log and I didn't find anything out of the ordinary.
 
Last edited:
M

mistwang

LiteSpeed Staff
#3
#3
check the external application statistics in the real time status report. If "Wait Q" become none-zero, you need to increase "PHP suEXEC Max conn" if PHP suEXEC is used.

Sometime eAccelerator will cause trouble for certain PHP version, turn it off may help.
 
M

MikeDVB

Well-Known Member
#4
#4
mistwang said:
check the external application statistics in the real time status report. If "Wait Q" become none-zero, you need to increase "PHP suEXEC Max conn" if PHP suEXEC is used.

Sometime eAccelerator will cause trouble for certain PHP version, turn it off may help.
Click to expand...
Not using eAccelerator so that's not an issue - I'll have to find the "Wait Q" and see. Is there a log file where "Wait Q" would be written to should it become non-zero?

I've upped the suPHP from 10 to 25 - if 10 was almost enough for 160+ simultaneous users and there being plenty of extra CPU/Ram on the system I think 25 should be fine. I'll watch it and update but I do believe that is likely the solution.
 
Last edited:
You must log in or register to reply here.
Top