optimization help

[Dani]

Now that we have finally bought LS "yay" I need some help regarding the configuration and if it can be pushed to serve everything just a little faster.
As we had apache before it did feel like it went alittle 'faster' but it also gave us alot of sql errors -> unable to connect and so it wasn't the perfect setup.

Now with LS it's going great except when the 'rush hour' begins as you can see on the graphs, the load gets 'really' high and sql gets slow. Images + sql + php is hosted on the same server. We get around 2300 online users at rush hour, our site is mainly a community + forum + photo albums etc.

But to keep a long story short here are specs that might be useful to know and current configuration.

Ubuntu dapper
PHP 5.2.3 - eaccelerator 0.9.5.2 (note I've also run the server with latest apc but haven't seen big differences so either one is ok with me)
mysql 5.0.22

cpu: Intel Xeon-Clovertown 5320-QuadCore 1.86GHz
ram: 6GB
2 hdd: I've balanced so that both are about max 1mb read/second so this is not the bottleneck.


LS config:

Memory I/O Buffer: 240M (not sure what it should be here)

Connection-> (somewhat played with to try to get better result)
Max Connections: 2000
Connection Timeout (secs): 15
Max Keep-Alive Requests: 15000
Smart Keep-Alive: No
Keep-Alive Timeout (secs): 5
Send Buffer Size (bytes): 0
Receive Buffer Size (bytes): 0
I/O Event Dispatcher: epoll (Linux 2.6 kernel)
Max Cached Small File Size (bytes): 4096
Total Small File Cache Size (bytes): 20M
Max MMAP File Size (bytes): 256K
Total MMAP Cache Size (bytes): 40M
Use sendfile(): Yes


Gzip-> (unchanged from default)
On
level 6 compression
Auto Update Static: Yes

Per Client Throttling-> (unchanged from default)
Static Requests/second 0
Dynamic Requests/second 0
Outbound Bandwidth (bytes/sec) 0
Inbound Bandwidth (bytes/sec) 0
Connection Soft Limit 10000
Connection Hard Limit 10000
Grace Period (sec) 15
Banned Period (sec) 300

External Application Resource->
CGI Daemon Socket Not Set
Max CGI Instances 20
Minimum UID 11
Minimum GID 10
Force GID Not Set
Priority 0
CPU Soft Limit (sec) 60
CPU Hard Limit (sec) 120
Memory Soft Limit (bytes) 250M
Memory Hard Limit (bytes) 300M
Process Soft Limit 700
Process Hard Limit 850



LiteSpeed API App Definition-> (all are default values from LS install except these)
MAx connections: 150
PHP_LSAPI_MAX_REQUESTS=700
PHP_LSAPI_CHILDREN=150
Instances=1


3 days cacti monitoring
http://img511.imageshack.us/img511/5606/cactial3.png


hope this helps you to help us.
thanks for reading

[mistwang]

Try the following:

level compression 1

Dynamic Requests/second 1
Connection Soft Limit 15
Connection Hard Limit 30

LiteSpeed API App Definition-> (all are default values from LS install except these)
MAx connections: 80
PHP_LSAPI_MAX_REQUESTS=700
PHP_LSAPI_CHILDREN=80

[Dani]

here are todays result, I had to increase the max connections and children for LS API since I got a lot off connection errors, but looking at the graphs it seems that they should be raised even more?

http://img98.imageshack.us/img98/18/cacti2ml0.png

Also the high load is a little tough to explain why that is. I do more and more cache optimizations to lower the cpu usage of the mysql server but the loads seems to keep rising for some reason. Last night I switched eaccelerator back to APC but got an even higher load, so I'm back to eaccelerator again.


thanks in advance for any help that we can get :)
Btw the latest ui fixes for the admin page made it really really fast. Great work there.

[mistwang]

I think you need to find some way to improve your php performance, or avoid generate same php page again, using various cache. like page cache, memcache, etc.

[viettechorg]

Instances=1

Does that mean you are running 1 CPU liecence?

Is the server serving mysql as well? So if, i think it's a good time to move mysql to second server.

[Dani]

Instances = 1 I think means that the php should have one parent dus handling the memory in a better way if I'm not mistaken.

And yes we're still running mysql on the server :) Optimized the php so that the load is peaked on 100 now and no more red LSAPI connections so far. It's all about the optimization of the code :)

[viettechorg]

I may think you have problem with I/O diskwait. I have a Forum about the same size as yours and after moveed mysql to second server it helps, both server running SCSI 15K.

btw, what liecence are you having? 1CPU or 2CPU? cause i am on trial and about to upgrade my server to clovertown 5310.

[mistwang]

It may help with high disk I/O wait if you mount the partition with "noatime"