LiteSpeed Support Forums - Configuring and testing litespeed cache Part 2

LiteSpeed Support Forums (http://www.litespeedtech.com/support/forum/index.php)

- LiteSpeed Cache (http://www.litespeedtech.com/support/forum/forumdisplay.php?f=30)

- - Configuring and testing litespeed cache Part 2 (http://www.litespeedtech.com/support/forum/showthread.php?t=4603)

Configuring and testing litespeed cache Part 2

Part 1 tests were flawed in that I tested Apache 2.2.3 with keepalives OFF. So this is round 2 tests with Apache 2.2.3 keepalives On thrown into compare with Litespeed 4.0.18 inbuilt caching feature. The Litespeed 4.0.18 cache feature might have a bug in not observing mod rewrite urls to cache only specified files which upgrading to 4.0.19 may fix. I'll update to Litespeed 4.0.19 and rerun tests as well.

Update: Upgraded from Litespeed 4.0.18 to 4.0.19 and fixed the cache feature bug see http://www.litespeedtech.com/support...4&postcount=11

Configurations tested

Apache 2.2.3 keepalives off
Apache 2.2.3 keepalives on
Apache 2.2.3 keepalives on + Varnish 2.1.4
Litespeed 4.0.18 no cache
Litespeed 4.0.18 no cache + Varnish 2.1.4
Litespeed 4.0.18 + inbuilt cache

Results

Part 1 tests illustrated a slightly different picture to what these updated tests had showed. The end result though was still the same though, Litespeed + inbuilt cache was up to 2x times faster than Apache + Varnish for simple test.php file apachebench runs.

Apache stand alone with keepalives enabled just edged out Litespeed no cached results at less <600 concurrency level. Past >600 concurrency level, Litespeed no cached average and minimum requests per second were much better. Note: Apache cpu load was much higher hitting into double digits by 5th run at >600 concurrency levels, while Litespeed cpu load was <1.5 all the way up to 1000 concurrency levels. Update: seems apachebench test configuration can play a big part, below tests were with 1000 requests tested, if you increase requests per test to 10000, litespeed starts to really shine more in stand alone nocache and cache tests - favouring even more, litespeed for scalability as traffic increases. Will retest later.
Apache + Varnish caching with default.vcl kept the same trend in just edging out Litespeed + Varnish caching when below <600 concurrency level. At 800 concurrency level, Litespeed + Varnish took the lead over Apache + Varnish. But at 1000 concurrency level not sure what happened as Litespeed + Varnish took a nose dive for the first 3 runs at 1/2 the rps compared to Apache + Varnish lowering the average to 6,163.51 rps compared to Apache + Varnish at 8,643.01 rps average.
A note on Varnish caching tests for part 2: Part 2 used a tuned Varnish configuration settings which consistently averages around + 500 rps more than untuned out of the box Varnish configuration.
Litespeed using inbuilt cache feature was the overall winner though with a comfortable lead. Litespeed + inbuilt cache was 4x times faster than Litespeed with no cache at 200 concurrency level and 4.5x times faster at 1000 concurrency level. Litespeed + inbuilt cache was up to 1.6x to 1.9x times faster than Apache + Varnish at concurrency levels 200 all the way up to 1000.

http://vbtechsupport.com/litespeed/b...21_2011_01.png

http://vbtechsupport.com/litespeed/b...21_2011_02.png

http://vbtechsupport.com/litespeed/b...21_2011_03.png

The test.php file used contains

Code:

<?php

header('CurrentTime: '.gmdate('D, d M Y H:i:s', time()).' GMT',true);



echo "time()=" . time() . "<br>";

echo "date()=" . date(DATE_RFC822) ."<br>";

?>

Apache httpd.conf contains

Code:

#disk cache

<IfModule mod_cache.c>

<IfModule mod_disk_cache.c>

CacheRoot /lscache/   

#CacheEnable disk /  

</IfModule>

</IfModule>

Litespeed 4.0.18 cache policy is set to

Code:

Enable Cache:Not set

Cache Expire Time (seconds): 120

Cache Request with Query String:Not set

Cache Request with Cookie:Not set

Cache Response with Cookie:Not set

Ignore Request Cache-Control:Not set

Ignore Response Cache-Control:Not set

.htaccess used in /var/www/html doc root for Litespeed 4.0.18 cache tests

Code:

RewriteEngine on

RewriteRule test.php - [E=Cache-Control:max-age=45]

ApacheBench command ran back to back 5x times

:for non-varnish tests:

Code:

ab -k -n 1000 -c 200 192.168.56.101/test.php

:for varnish tests on port 8888:

Code:

ab -k -n 1000 -c 200 192.168.56.101:8888/test.php

Varnish 2.1.4 configuration settings used

Code:

/etc/sysconfig/varnish



# My Advanced configuration

# Main configuration file. You probably want to change it :)

VARNISH_VCL_CONF=/etc/varnish/default.vcl



# Default address and port to bind to

# Blank address means all IPv4 and IPv6 interfaces, otherwise specify

# a host name, an IPv4 dotted quad, or an IPv6 address in brackets.

VARNISH_LISTEN_ADDRESS=

VARNISH_LISTEN_PORT=8888



# Telnet admin interface listen address and port

VARNISH_ADMIN_LISTEN_ADDRESS=127.0.0.1

VARNISH_ADMIN_LISTEN_PORT=2222



# The minimum number of worker threads to start

VARNISH_MIN_THREADS=1



# The Maximum number of worker threads to start

VARNISH_MAX_THREADS=1000



# Idle timeout for worker threads

VARNISH_THREAD_TIMEOUT=120



# Cache file location

VARNISH_STORAGE_FILE=/var/lib/varnish/varnish_storage.bin



# Cache file size: in bytes, optionally using k / M / G / T suffix,      

# or in percentage of available disk space using the &#37; suffix.

VARNISH_STORAGE_SIZE=64M



# Backend storage specification             

VARNISH_STORAGE="file,${VARNISH_STORAGE_FILE},${VARNISH_STORAGE_SIZE}"



# Default TTL used when the backend does not specify one

VARNISH_TTL=120                  

            

# DAEMON_OPTS is used by the init script.  If you add or remove options, make

# sure you update this section, too.

DAEMON_OPTS="-a ${VARNISH_LISTEN_ADDRESS}:${VARNISH_LISTEN_PORT} \

             -f ${VARNISH_VCL_CONF} \

             -T ${VARNISH_ADMIN_LISTEN_ADDRESS}:${VARNISH_ADMIN_LISTEN_PORT} \

             -t ${VARNISH_TTL} \

             -w ${VARNISH_MIN_THREADS},${VARNISH_MAX_THREADS},${VARNISH_THREAD_TIMEOUT} \

             -u varnish -g varnish \

             -s ${VARNISH_STORAGE} \

             -p thread_pool_min=300 \

             -p thread_pool_max=2000 \

             -p thread_pools=2 \

             -p listen_depth=4096 \

             -p session_linger=25/100/150 \

             -p lru_interval=2 \

             -p thread_pool_add_delay=2 \

             -p cli_timeout=10"

Server Configuration:

VirtualBox Guest - CentOS 5.5 64bit
Xeon W3540 @3408Mhz - assigned 2 cpu cores
1GB memory allocated system memory + 64MB GPU memory @DDR3-1550Mhz 9-9-9-24
20GB allocated (640GB Samsung SATAII OS Disk)
Apache 2.2.3 Prefork, PHP 5.3.5 (mod_php), MariaDB 5.2.4, Memcached 1.4.5, Varnish 2.1.4
Litespeed 4.0.18, PHP 5.3.4 (LSAPI v5.5), MariaDB 5.2.4, Memcached 1.4.5, Varnish 2.1.4
2.6.18-194.32.1.el5 #1 SMP
Disk partitions set to noatime
Memcached 1.4.5 = 2x 16MB instances
Varnish 2.1.4 = 64MB size

References:

mistwang and NiteWave just a question. In above tests, Litespeed 4.0.18 had smart keep alives enabled. Looking up the function it doesn't mention specifically how it treats php files ?

Quote:

Smart Keep-Alive
Description: Specifies whether to turn on Smart Keep-Alive. This option is effective only if Max Keep-Alive Requests > 1. If enabled, you can also enable/disable it at virtual host level. Smart keep-alive will only establish keep-alive connections for requests of JavaScript, CSS Style Sheet and image files. For html pages, connection will not be kept alive. This will help serve more users more efficiently. Normally a web page contains multiple images and scripts that will be cached by the browsers after initial request. It is more efficient to send those non-html static files through a single keep-alive connection and have the text/html file send through another non-keep-alive connection. This method will reduce idle connections and in turn increase capacity to handle more concurrent requests and users.

Update: Looks like Litespeed with Smart Keep-Alive OFF (which is default) is better than with it On.

http://vbtechsupport.com/litespeed/b..._skaoff_01.png

http://vbtechsupport.com/litespeed/b..._skaoff_02.png

http://vbtechsupport.com/litespeed/b..._skaoff_03.png

http://vbtechsupport.com/litespeed/b..._skaoff_04.png

And the interesting results,

Apache + Varnish
vs
Litespeed Cache with Smart Keep Alives On vs Off

http://vbtechsupport.com/litespeed/b..._skaoff_05.png

http://vbtechsupport.com/litespeed/b..._skaoff_06.png

smart keepalive will close connection after serving a response with MIME type text/* regardless it is from PHP or static file. So, "Smart Keepalive" should be off when you did the benchmark tests.

add environment
LSAPI_ACCEPT_NOTIFY=1
LSAPI_AVOID_FORK=1

to lsphp5 external app should boost Litespeed no-cache results.

I wonder if you use mod_php or suPHP with Apache?
If uses mod_php, make sure "PHP suEXEC" is off to match Apache security model.

hmmm you should enable multi-quote feature in vB so i can quote you properly :)

Quote:

Originally Posted by mistwang (Post 22997)

smart keepalive will close connection after serving a response with MIME type text/* regardless it is from PHP or static file. So, "Smart Keepalive" should be off when you did the benchmark tests.

i see, yup 2nd post above has benchmarks redone with smart keep alive off with a nice boost especially at >1000 concurrency level.

Quote:

Originally Posted by mistwang (Post 23000)

add environment
LSAPI_ACCEPT_NOTIFY=1
LSAPI_AVOID_FORK=1

to lsphp5 external app should boost Litespeed no-cache results.

Ooooh additional tweaks to try - will do :)

Quote:

Originally Posted by mistwang (Post 23002)

I wonder if you use mod_php or suPHP with Apache?
If uses mod_php, make sure "PHP suEXEC" is off to match Apache security model.

Yeah Apache is mod_php with PHP suExec disabled as well as disabled when loading Apache conf via litespeed :)

Tried these 2 suggested values but getting slower performance as opposed to the defaults as per http://www.litespeedtech.com/php-litespeed-sapi.html

only benched the no cache config

http://vbtechsupport.com/litespeed/b..._phptwk_01.png

Code:

LSAPI_ACCEPT_NOTIFY=1

LSAPI_AVOID_FORK=1

Quote:

# LSAPI_AVOID_FORK (default: 0)
LSAPI_AVOID_FORK specifies the policy of the internal process manager in "Self Managed Mode". When set to 0, the internal process manager will stop and start children process on demand to save system resource. This is preferred in a shared hosting environment. When set to 1, the internal process manager will try to avoid freqently stopping and starting children process. This might be preferred in a dedicate hosting environment.

# LSAPI_ACCEPT_NOTIFY
By default a LSAPI application will send back a notification packet whenever a request has been received. Since PHP LSAPI 5.0 release, it can be changed to only notify server for newly established connection by setting this option to '1'. It is recommended for LiteSpeed Web Server 4.1 release and later to gain more performance.

LSAPI PHP5 addition settings

http://vbtechsupport.com/litespeed/b...figuration.png

I'll upgrade to Litespeed 4.0.19 now to see if can fix the cache options issues. Will also update LSAPI PHP from PHP 5.3.4 to 5.3.5 to match Apache's PHP 5.3.5 version. Seem to have run into an error building PHP 5.3.5 http://www.litespeedtech.com/support...3014#post23014

It is about the same, you can also change
"Max connections", "PHP_LSAPI_CHILDREN", to 10 and 100, see which one does better.
PHP_LSAPI_MAX_REQUESTS can be increased to a larger number.

How many total requests served for each run? I think it should be >1000.

There is likely some factor affect the consecutive run of the benchmark tests, there should not be huge differences between them. Maybe the time you wait between each run has impact on this as kernel takes some time to recover from massive amount of sockets created from previous run?

I may try to reproduce this in our lab to figure out why if I get a chance.

How many total requests served for each run, i have the raw ab numbers for latest runs here

i.e for litespeed cached + smart keep alive off

Code:

ab -k -n 1000 -c 200 192.168.56.101/test.php

This is ApacheBench, Version 2.0.40-dev <$Revision: 1.146 $> apache-2.0

Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/

Copyright 2006 The Apache Software Foundation, http://www.apache.org/



Benchmarking 192.168.56.101 (be patient)

Completed 100 requests

Completed 200 requests

Completed 300 requests

Completed 400 requests

Completed 500 requests

Completed 600 requests

Completed 700 requests

Completed 800 requests

Completed 900 requests

Finished 1000 requests





Server Software:        LiteSpeed

Server Hostname:        192.168.56.101

Server Port:            80



Document Path:          /test.php

Document Length:        63 bytes



Concurrency Level:      200

Time taken for tests:   0.64157 seconds

Complete requests:      1000

Failed requests:        0

Write errors:           0

Keep-Alive requests:    993

Total transferred:      322741 bytes

HTML transferred:       63000 bytes

Requests per second:    15586.76 [#/sec] (mean)

Time per request:       12.831 [ms] (mean)

Time per request:       0.064 [ms] (mean, across all concurrent requests)

Transfer rate:          4909.83 [Kbytes/sec] received



Connection Times (ms)

              min  mean[+/-sd] median   max

Connect:        0    0   0.0      0       0

Processing:     0    0   3.5      0      55

Waiting:        0    0   3.4      0      55

Total:          0    0   3.5      0      55



Percentage of the requests served within a certain time (ms)

  50&#37;      0

  66%      0

  75%      0

  80%      0

  90%      0

  95%      0

  98%      1

  99%      5

 100%     55 (longest request)

Yeah i'm running the ab runs back to back not much wait time. Could also be virtualized server overhead ?

Will try the other php settings once i can fix PHP 5.3.5 build errors http://www.litespeedtech.com/support...3014#post23014 phpize not where it's suppose to be heh.