Configuring and testing litespeed cache Part 2

[eva2000]

Part 1 tests were flawed in that I tested Apache 2.2.3 with keepalives OFF. So this is round 2 tests with Apache 2.2.3 keepalives On thrown into compare with Litespeed 4.0.18 inbuilt caching feature. The Litespeed 4.0.18 cache feature might have a bug in not observing mod rewrite urls to cache only specified files which upgrading to 4.0.19 may fix. I'll update to Litespeed 4.0.19 and rerun tests as well.

Update: Upgraded from Litespeed 4.0.18 to 4.0.19 and fixed the cache feature bug see http://www.litespeedtech.com/support...4&postcount=11

Configurations tested

• Apache 2.2.3 keepalives off
• Apache 2.2.3 keepalives on
• Apache 2.2.3 keepalives on + Varnish 2.1.4
• Litespeed 4.0.18 no cache
• Litespeed 4.0.18 no cache + Varnish 2.1.4
• Litespeed 4.0.18 + inbuilt cache

Results

Part 1 tests illustrated a slightly different picture to what these updated tests had showed. The end result though was still the same though, Litespeed + inbuilt cache was up to 2x times faster than Apache + Varnish for simple test.php file apachebench runs.

1. Apache stand alone with keepalives enabled just edged out Litespeed no cached results at less <600 concurrency level. Past >600 concurrency level, Litespeed no cached average and minimum requests per second were much better. Note: Apache cpu load was much higher hitting into double digits by 5th run at >600 concurrency levels, while Litespeed cpu load was <1.5 all the way up to 1000 concurrency levels. Update: seems apachebench test configuration can play a big part, below tests were with 1000 requests tested, if you increase requests per test to 10000, litespeed starts to really shine more in stand alone nocache and cache tests - favouring even more, litespeed for scalability as traffic increases. Will retest later.
   
2. Apache + Varnish caching with default.vcl kept the same trend in just edging out Litespeed + Varnish caching when below <600 concurrency level. At 800 concurrency level, Litespeed + Varnish took the lead over Apache + Varnish. But at 1000 concurrency level not sure what happened as Litespeed + Varnish took a nose dive for the first 3 runs at 1/2 the rps compared to Apache + Varnish lowering the average to 6,163.51 rps compared to Apache + Varnish at 8,643.01 rps average.
   
3. A note on Varnish caching tests for part 2: Part 2 used a tuned Varnish configuration settings which consistently averages around + 500 rps more than untuned out of the box Varnish configuration.
   
4. Litespeed using inbuilt cache feature was the overall winner though with a comfortable lead. Litespeed + inbuilt cache was 4x times faster than Litespeed with no cache at 200 concurrency level and 4.5x times faster at 1000 concurrency level. Litespeed + inbuilt cache was up to 1.6x to 1.9x times faster than Apache + Varnish at concurrency levels 200 all the way up to 1000.

The test.php file used contains

<?php
header('CurrentTime: '.gmdate('D, d M Y H:i:s', time()).' GMT',true);

echo "time()=" . time() . "<br>";
echo "date()=" . date(DATE_RFC822) ."<br>";
?>

Apache httpd.conf contains

#disk cache
<IfModule mod_cache.c>
<IfModule mod_disk_cache.c>
CacheRoot /lscache/   
#CacheEnable disk /  
</IfModule>
</IfModule>

Litespeed 4.0.18 cache policy is set to

Enable Cache:Not set
Cache Expire Time (seconds): 120
Cache Request with Query String:Not set
Cache Request with Cookie:Not set
Cache Response with Cookie:Not set
Ignore Request Cache-Control:Not set
Ignore Response Cache-Control:Not set

.htaccess used in /var/www/html doc root for Litespeed 4.0.18 cache tests

RewriteEngine on
RewriteRule test.php - [E=Cache-Control:max-age=45]

ApacheBench command ran back to back 5x times

:for non-varnish tests:

ab -k -n 1000 -c 200 192.168.56.101/test.php

:for varnish tests on port 8888:

ab -k -n 1000 -c 200 192.168.56.101:8888/test.php

Varnish 2.1.4 configuration settings used

/etc/sysconfig/varnish

# My Advanced configuration
# Main configuration file. You probably want to change it :)
VARNISH_VCL_CONF=/etc/varnish/default.vcl

# Default address and port to bind to
# Blank address means all IPv4 and IPv6 interfaces, otherwise specify
# a host name, an IPv4 dotted quad, or an IPv6 address in brackets.
VARNISH_LISTEN_ADDRESS=
VARNISH_LISTEN_PORT=8888

# Telnet admin interface listen address and port
VARNISH_ADMIN_LISTEN_ADDRESS=127.0.0.1
VARNISH_ADMIN_LISTEN_PORT=2222

# The minimum number of worker threads to start
VARNISH_MIN_THREADS=1

# The Maximum number of worker threads to start
VARNISH_MAX_THREADS=1000

# Idle timeout for worker threads
VARNISH_THREAD_TIMEOUT=120

# Cache file location
VARNISH_STORAGE_FILE=/var/lib/varnish/varnish_storage.bin

# Cache file size: in bytes, optionally using k / M / G / T suffix,      
# or in percentage of available disk space using the % suffix.
VARNISH_STORAGE_SIZE=64M

# Backend storage specification             
VARNISH_STORAGE="file,${VARNISH_STORAGE_FILE},${VARNISH_STORAGE_SIZE}"

# Default TTL used when the backend does not specify one
VARNISH_TTL=120                  
            
# DAEMON_OPTS is used by the init script.  If you add or remove options, make
# sure you update this section, too.
DAEMON_OPTS="-a ${VARNISH_LISTEN_ADDRESS}:${VARNISH_LISTEN_PORT} \
             -f ${VARNISH_VCL_CONF} \
             -T ${VARNISH_ADMIN_LISTEN_ADDRESS}:${VARNISH_ADMIN_LISTEN_PORT} \
             -t ${VARNISH_TTL} \
             -w ${VARNISH_MIN_THREADS},${VARNISH_MAX_THREADS},${VARNISH_THREAD_TIMEOUT} \
             -u varnish -g varnish \
             -s ${VARNISH_STORAGE} \
             -p thread_pool_min=300 \
             -p thread_pool_max=2000 \
             -p thread_pools=2 \
             -p listen_depth=4096 \
             -p session_linger=25/100/150 \
             -p lru_interval=2 \
             -p thread_pool_add_delay=2 \
             -p cli_timeout=10"

Server Configuration:

• VirtualBox Guest - CentOS 5.5 64bit
• Xeon W3540 @3408Mhz - assigned 2 cpu cores
• 1GB memory allocated system memory + 64MB GPU memory @DDR3-1550Mhz 9-9-9-24
• 20GB allocated (640GB Samsung SATAII OS Disk)
• Apache 2.2.3 Prefork, PHP 5.3.5 (mod_php), MariaDB 5.2.4, Memcached 1.4.5, Varnish 2.1.4
• Litespeed 4.0.18, PHP 5.3.4 (LSAPI v5.5), MariaDB 5.2.4, Memcached 1.4.5, Varnish 2.1.4
• 2.6.18-194.32.1.el5 #1 SMP
• Disk partitions set to noatime
• Memcached 1.4.5 = 2x 16MB instances
• Varnish 2.1.4 = 64MB size

References:

• Content-Aware Caching
• vBulletin Guest Cache with LiteSpeed
• Configuring and testing litespeed cache Part 1

[eva2000]

mistwang and NiteWave just a question. In above tests, Litespeed 4.0.18 had smart keep alives enabled. Looking up the function it doesn't mention specifically how it treats php files ?

Smart Keep-Alive
Description: Specifies whether to turn on Smart Keep-Alive. This option is effective only if Max Keep-Alive Requests > 1. If enabled, you can also enable/disable it at virtual host level. Smart keep-alive will only establish keep-alive connections for requests of JavaScript, CSS Style Sheet and image files. For html pages, connection will not be kept alive. This will help serve more users more efficiently. Normally a web page contains multiple images and scripts that will be cached by the browsers after initial request. It is more efficient to send those non-html static files through a single keep-alive connection and have the text/html file send through another non-keep-alive connection. This method will reduce idle connections and in turn increase capacity to handle more concurrent requests and users.

Update: Looks like Litespeed with Smart Keep-Alive OFF (which is default) is better than with it On.









And the interesting results,

Apache + Varnish
vs
Litespeed Cache with Smart Keep Alives On vs Off

[mistwang]

smart keepalive will close connection after serving a response with MIME type text/* regardless it is from PHP or static file. So, "Smart Keepalive" should be off when you did the benchmark tests.

[mistwang]

add environment
LSAPI_ACCEPT_NOTIFY=1
LSAPI_AVOID_FORK=1

to lsphp5 external app should boost Litespeed no-cache results.

[mistwang]

I wonder if you use mod_php or suPHP with Apache?
If uses mod_php, make sure "PHP suEXEC" is off to match Apache security model.

[eva2000]

hmmm you should enable multi-quote feature in vB so i can quote you properly

Originally Posted by mistwang

smart keepalive will close connection after serving a response with MIME type text/* regardless it is from PHP or static file. So, "Smart Keepalive" should be off when you did the benchmark tests.

i see, yup 2nd post above has benchmarks redone with smart keep alive off with a nice boost especially at >1000 concurrency level.

Originally Posted by mistwang

add environment
LSAPI_ACCEPT_NOTIFY=1
LSAPI_AVOID_FORK=1

to lsphp5 external app should boost Litespeed no-cache results.

Ooooh additional tweaks to try - will do

Originally Posted by mistwang

I wonder if you use mod_php or suPHP with Apache?
If uses mod_php, make sure "PHP suEXEC" is off to match Apache security model.

Yeah Apache is mod_php with PHP suExec disabled as well as disabled when loading Apache conf via litespeed

[eva2000]

Tried these 2 suggested values but getting slower performance as opposed to the defaults as per http://www.litespeedtech.com/php-litespeed-sapi.html

only benched the no cache config

LSAPI_ACCEPT_NOTIFY=1
LSAPI_AVOID_FORK=1

# LSAPI_AVOID_FORK (default: 0)
LSAPI_AVOID_FORK specifies the policy of the internal process manager in "Self Managed Mode". When set to 0, the internal process manager will stop and start children process on demand to save system resource. This is preferred in a shared hosting environment. When set to 1, the internal process manager will try to avoid freqently stopping and starting children process. This might be preferred in a dedicate hosting environment.

# LSAPI_ACCEPT_NOTIFY
By default a LSAPI application will send back a notification packet whenever a request has been received. Since PHP LSAPI 5.0 release, it can be changed to only notify server for newly established connection by setting this option to '1'. It is recommended for LiteSpeed Web Server 4.1 release and later to gain more performance.

LSAPI PHP5 addition settings

[eva2000]

I'll upgrade to Litespeed 4.0.19 now to see if can fix the cache options issues. Will also update LSAPI PHP from PHP 5.3.4 to 5.3.5 to match Apache's PHP 5.3.5 version. Seem to have run into an error building PHP 5.3.5 http://www.litespeedtech.com/support...3014#post23014

[mistwang]

It is about the same, you can also change
"Max connections", "PHP_LSAPI_CHILDREN", to 10 and 100, see which one does better.
PHP_LSAPI_MAX_REQUESTS can be increased to a larger number.

How many total requests served for each run? I think it should be >1000.

There is likely some factor affect the consecutive run of the benchmark tests, there should not be huge differences between them. Maybe the time you wait between each run has impact on this as kernel takes some time to recover from massive amount of sockets created from previous run?

I may try to reproduce this in our lab to figure out why if I get a chance.

[eva2000]

How many total requests served for each run, i have the raw ab numbers for latest runs here

i.e for litespeed cached + smart keep alive off

ab -k -n 1000 -c 200 192.168.56.101/test.php
This is ApacheBench, Version 2.0.40-dev <$Revision: 1.146 $> apache-2.0
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Copyright 2006 The Apache Software Foundation, http://www.apache.org/

Benchmarking 192.168.56.101 (be patient)
Completed 100 requests
Completed 200 requests
Completed 300 requests
Completed 400 requests
Completed 500 requests
Completed 600 requests
Completed 700 requests
Completed 800 requests
Completed 900 requests
Finished 1000 requests


Server Software:        LiteSpeed
Server Hostname:        192.168.56.101
Server Port:            80

Document Path:          /test.php
Document Length:        63 bytes

Concurrency Level:      200
Time taken for tests:   0.64157 seconds
Complete requests:      1000
Failed requests:        0
Write errors:           0
Keep-Alive requests:    993
Total transferred:      322741 bytes
HTML transferred:       63000 bytes
Requests per second:    15586.76 [#/sec] (mean)
Time per request:       12.831 [ms] (mean)
Time per request:       0.064 [ms] (mean, across all concurrent requests)
Transfer rate:          4909.83 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.0      0       0
Processing:     0    0   3.5      0      55
Waiting:        0    0   3.4      0      55
Total:          0    0   3.5      0      55

Percentage of the requests served within a certain time (ms)
  50&#37;      0
  66%      0
  75%      0
  80%      0
  90%      0
  95%      0
  98%      1
  99%      5
 100%     55 (longest request)

Yeah i'm running the ab runs back to back not much wait time. Could also be virtualized server overhead ?

Will try the other php settings once i can fix PHP 5.3.5 build errors http://www.litespeedtech.com/support...3014#post23014 phpize not where it's suppose to be heh.