Configuring and testing litespeed cache Part 2

[eva2000]

Part 1 tests were flawed in that I tested Apache 2.2.3 with keepalives OFF. So this is round 2 tests with Apache 2.2.3 keepalives On thrown into compare with Litespeed 4.0.18 inbuilt caching feature. The Litespeed 4.0.18 cache feature might have a bug in not observing mod rewrite urls to cache only specified files which upgrading to 4.0.19 may fix. I'll update to Litespeed 4.0.19 and rerun tests as well.

Update: Upgraded from Litespeed 4.0.18 to 4.0.19 and fixed the cache feature bug see http://www.litespeedtech.com/support...4&postcount=11

Configurations tested

• Apache 2.2.3 keepalives off
• Apache 2.2.3 keepalives on
• Apache 2.2.3 keepalives on + Varnish 2.1.4
• Litespeed 4.0.18 no cache
• Litespeed 4.0.18 no cache + Varnish 2.1.4
• Litespeed 4.0.18 + inbuilt cache

Results

Part 1 tests illustrated a slightly different picture to what these updated tests had showed. The end result though was still the same though, Litespeed + inbuilt cache was up to 2x times faster than Apache + Varnish for simple test.php file apachebench runs.

1. Apache stand alone with keepalives enabled just edged out Litespeed no cached results at less <600 concurrency level. Past >600 concurrency level, Litespeed no cached average and minimum requests per second were much better. Note: Apache cpu load was much higher hitting into double digits by 5th run at >600 concurrency levels, while Litespeed cpu load was <1.5 all the way up to 1000 concurrency levels. Update: seems apachebench test configuration can play a big part, below tests were with 1000 requests tested, if you increase requests per test to 10000, litespeed starts to really shine more in stand alone nocache and cache tests - favouring even more, litespeed for scalability as traffic increases. Will retest later.
   
2. Apache + Varnish caching with default.vcl kept the same trend in just edging out Litespeed + Varnish caching when below <600 concurrency level. At 800 concurrency level, Litespeed + Varnish took the lead over Apache + Varnish. But at 1000 concurrency level not sure what happened as Litespeed + Varnish took a nose dive for the first 3 runs at 1/2 the rps compared to Apache + Varnish lowering the average to 6,163.51 rps compared to Apache + Varnish at 8,643.01 rps average.
   
3. A note on Varnish caching tests for part 2: Part 2 used a tuned Varnish configuration settings which consistently averages around + 500 rps more than untuned out of the box Varnish configuration.
   
4. Litespeed using inbuilt cache feature was the overall winner though with a comfortable lead. Litespeed + inbuilt cache was 4x times faster than Litespeed with no cache at 200 concurrency level and 4.5x times faster at 1000 concurrency level. Litespeed + inbuilt cache was up to 1.6x to 1.9x times faster than Apache + Varnish at concurrency levels 200 all the way up to 1000.

The test.php file used contains

<?php
header('CurrentTime: '.gmdate('D, d M Y H:i:s', time()).' GMT',true);

echo "time()=" . time() . "<br>";
echo "date()=" . date(DATE_RFC822) ."<br>";
?>

Apache httpd.conf contains

#disk cache
<IfModule mod_cache.c>
<IfModule mod_disk_cache.c>
CacheRoot /lscache/   
#CacheEnable disk /  
</IfModule>
</IfModule>

Litespeed 4.0.18 cache policy is set to

Enable Cache:Not set
Cache Expire Time (seconds): 120
Cache Request with Query String:Not set
Cache Request with Cookie:Not set
Cache Response with Cookie:Not set
Ignore Request Cache-Control:Not set
Ignore Response Cache-Control:Not set

.htaccess used in /var/www/html doc root for Litespeed 4.0.18 cache tests

RewriteEngine on
RewriteRule test.php - [E=Cache-Control:max-age=45]

ApacheBench command ran back to back 5x times

:for non-varnish tests:

ab -k -n 1000 -c 200 192.168.56.101/test.php

:for varnish tests on port 8888:

ab -k -n 1000 -c 200 192.168.56.101:8888/test.php

Varnish 2.1.4 configuration settings used

/etc/sysconfig/varnish

# My Advanced configuration
# Main configuration file. You probably want to change it :)
VARNISH_VCL_CONF=/etc/varnish/default.vcl

# Default address and port to bind to
# Blank address means all IPv4 and IPv6 interfaces, otherwise specify
# a host name, an IPv4 dotted quad, or an IPv6 address in brackets.
VARNISH_LISTEN_ADDRESS=
VARNISH_LISTEN_PORT=8888

# Telnet admin interface listen address and port
VARNISH_ADMIN_LISTEN_ADDRESS=127.0.0.1
VARNISH_ADMIN_LISTEN_PORT=2222

# The minimum number of worker threads to start
VARNISH_MIN_THREADS=1

# The Maximum number of worker threads to start
VARNISH_MAX_THREADS=1000

# Idle timeout for worker threads
VARNISH_THREAD_TIMEOUT=120

# Cache file location
VARNISH_STORAGE_FILE=/var/lib/varnish/varnish_storage.bin

# Cache file size: in bytes, optionally using k / M / G / T suffix,      
# or in percentage of available disk space using the % suffix.
VARNISH_STORAGE_SIZE=64M

# Backend storage specification             
VARNISH_STORAGE="file,${VARNISH_STORAGE_FILE},${VARNISH_STORAGE_SIZE}"

# Default TTL used when the backend does not specify one
VARNISH_TTL=120                  
            
# DAEMON_OPTS is used by the init script.  If you add or remove options, make
# sure you update this section, too.
DAEMON_OPTS="-a ${VARNISH_LISTEN_ADDRESS}:${VARNISH_LISTEN_PORT} \
             -f ${VARNISH_VCL_CONF} \
             -T ${VARNISH_ADMIN_LISTEN_ADDRESS}:${VARNISH_ADMIN_LISTEN_PORT} \
             -t ${VARNISH_TTL} \
             -w ${VARNISH_MIN_THREADS},${VARNISH_MAX_THREADS},${VARNISH_THREAD_TIMEOUT} \
             -u varnish -g varnish \
             -s ${VARNISH_STORAGE} \
             -p thread_pool_min=300 \
             -p thread_pool_max=2000 \
             -p thread_pools=2 \
             -p listen_depth=4096 \
             -p session_linger=25/100/150 \
             -p lru_interval=2 \
             -p thread_pool_add_delay=2 \
             -p cli_timeout=10"

Server Configuration:

• VirtualBox Guest - CentOS 5.5 64bit
• Xeon W3540 @3408Mhz - assigned 2 cpu cores
• 1GB memory allocated system memory + 64MB GPU memory @DDR3-1550Mhz 9-9-9-24
• 20GB allocated (640GB Samsung SATAII OS Disk)
• Apache 2.2.3 Prefork, PHP 5.3.5 (mod_php), MariaDB 5.2.4, Memcached 1.4.5, Varnish 2.1.4
• Litespeed 4.0.18, PHP 5.3.4 (LSAPI v5.5), MariaDB 5.2.4, Memcached 1.4.5, Varnish 2.1.4
• 2.6.18-194.32.1.el5 #1 SMP
• Disk partitions set to noatime
• Memcached 1.4.5 = 2x 16MB instances
• Varnish 2.1.4 = 64MB size

References:

• Content-Aware Caching
• vBulletin Guest Cache with LiteSpeed
• Configuring and testing litespeed cache Part 1