LiteSpeed VS Unixy Varnish - The Truth!

[IPMAN]

Hello, Dear LiteSpeeders

I think its about time we got down to some serious public testing with this new varnish plugin by Unixy and not an apples to oranges comparison which is slowly getting around the forum's

Whiles you sort between you all who will do what, I will hunt down a decent provider who will lay on a good dedicated box for this test for acouple of day's, Hopefully we,ll beable to test other configurations against litespeed.

Whos going to do the configurations of the software for testing?

Whats the best configuration for testing?

Thanks,

 

[mistwang]

Any of our staff members should be able to help with the configuration.
We have published some guides for popular web applications, you can take a look at

http://www.litespeedtech.com/support/forum/forumdisplay.php?f=30

more to come.

 

[IPMAN]

Hello,

Just an update:

I have managed to get a dedicated server but its still in the provisioning process so I dont know how long its going to take. (Hopefully Soon)

I have also been offered a VPS so we might beable to run these tests in both the dedicated and virtual environment.

Spec's:

Dual Core Xeon
2GB Ram
500GB Sata Drive

Testing:

Varnish + WP SuperCache on
Varnish + WP SuperCache off
Varnish + Magento Store
LiteSpeed + WP SuperCache on
LiteSpeed + WP SuperCache off
LiteSpeed page cache on + SuperCache off
LiteSpeed page cache on + Magento Store

I am looking for a 3rd Party to run the testing, Anyone interested who can be trusted?

If we can get this lot of testing done in a timely manner, We could 'derail' and take a slight decourse and test litespeed against other things such as nginx, apache, etc

Thanks for reading and hope this interests you all

 

[mistwang]

Another test suggestion.

serving multiple large media files total size around 2GB or more.
LiteSpeed with cache on/off, Apache + varnish.

 

[IPMAN]

UPDATE: Let's Start This!

Hello,

This is an update on my progress to get this testing started.

Sadly the server which I purchased is still in the provisioning process.

But all is not lost because the VPS which I was offered has been accepted, provisioned and all details sent to me, Many Thanks to Dan from EZPZ Hosting for donating this monster of a VPS which is located on one of his Dev Node's (Note: Nothing critical is on the node)

This VPS has 6GB Ram which is plenty ample enough, Am sure Dan will chip-in on the specs of the main node.

Since no 3rd party's have stepped up to the mark of doing this testing someone from Litespeed will kindly kick this public show off, Once the testing is underway am sure people will step up, Remember only trusted people will be allowed to undertake the testing.

Let’s Get This Show Started!

 

[IPMAN]

Some Results - LSWS is 50% faster than varnish

Hello,

After some extensive setting up we are ready to publish some results.

Sample ab benchmarks are as follows:

Varnish on port 80 (cached)
Apache on port 82 (non cached)
LSWS on port 3082 (cache enabled)

LiteSpeed:

root@test [~]# ab -n 2000 -c 200 http://IPGOESHERE:3082/phpinfo.php
This is ApacheBench, Version 2.0.40-dev <$Revision: 1.146 $> apache-2.0
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Copyright 2006 The Apache Software Foundation, http://www.apache.org/

Benchmarking IPGOESHERE (be patient)
Completed 200 requests
Completed 400 requests
Completed 600 requests
Completed 800 requests
Completed 1000 requests
Completed 1200 requests
Completed 1400 requests
Completed 1600 requests
Completed 1800 requests
Finished 2000 requests


Server Software: LiteSpeed
Server Port: 3082

Document Path: /phpinfo.php
Document Length: 34243 bytes

Concurrency Level: 200
Time taken for tests: 0.260562 seconds
Complete requests: 2000
Failed requests: 0
Write errors: 0
Total transferred: 68971302 bytes
HTML transferred: 68588729 bytes
Requests per second: 7675.72 [#/sec] (mean)
Time per request: 26.056 [ms] (mean)
Time per request: 0.130 [ms] (mean, across all concurrent requests)
Transfer rate: 258495.09 [Kbytes/sec] received

Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 2 2.7 2 16
Processing: 8 22 3.9 23 39
Waiting: 1 4 2.9 4 21
Total: 9 24 5.2 25 51

Percentage of the requests served within a certain time (ms)
50% 25
66% 25
75% 26
80% 27
90% 29
95% 32
98% 40
99% 43
100% 51 (longest request)

Varnish:

root@test [~]# ab -n 2000 -c 200 http://IPGOESHERE:80/phpinfo.php
This is ApacheBench, Version 2.0.40-dev <$Revision: 1.146 $> apache-2.0
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Copyright 2006 The Apache Software Foundation, http://www.apache.org/

Benchmarking IPGOESHERE (be patient)
Completed 200 requests
Completed 400 requests
Completed 600 requests
Completed 800 requests
Completed 1000 requests
Completed 1200 requests
Completed 1400 requests
Completed 1600 requests
Completed 1800 requests
Finished 2000 requests


Server Software: Varnish
Server Port: 80

Document Path: /phpinfo.php
Document Length: 39265 bytes

Concurrency Level: 200
Time taken for tests: 0.414206 seconds
Complete requests: 2000
Failed requests: 0
Write errors: 0
Total transferred: 79172863 bytes
HTML transferred: 78530000 bytes
Requests per second: 4828.52 [#/sec] (mean)
Time per request: 41.421 [ms] (mean)
Time per request: 0.207 [ms] (mean, across all concurrent requests)
Transfer rate: 186663.16 [Kbytes/sec] received

Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 13 9.2 14 32
Processing: 7 26 3.7 26 69
Waiting: 0 11 6.9 12 30
Total: 8 40 7.4 40 73

Percentage of the requests served within a certain time (ms)
50% 40
66% 43
75% 46
80% 47
90% 50
95% 52
98% 53
99% 54
100% 73 (longest request)

As you can see LSWS is 50% faster than varnish!

LiteSpeed:
Time per request: 26.056 [ms] (mean)
Time per request: 0.130 [ms] (mean, across all concurrent requests)

Varnish:
Time per request: 41.421 [ms] (mean)
Time per request: 0.207 [ms] (mean, across all concurrent requests)

Apache:

root@test [~]# ab -n 2000 -c 200 http://IPGOESHERE:82/phpinfo.php
This is ApacheBench, Version 2.0.40-dev <$Revision: 1.146 $> apache-2.0
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Copyright 2006 The Apache Software Foundation, http://www.apache.org/

Benchmarking IPGOESHERE (be patient)
Completed 200 requests
Completed 400 requests
Completed 600 requests
Completed 800 requests
Completed 1000 requests
Completed 1200 requests
Completed 1400 requests
Completed 1600 requests
Completed 1800 requests
Finished 2000 requests


Server Software: Apache/2.0.63
Server Port: 82

Document Path: /phpinfo.php
Document Length: 38773 bytes

Concurrency Level: 200
Time taken for tests: 12.909277 seconds
Complete requests: 2000
Failed requests: 0
Write errors: 0
Total transferred: 78232712 bytes
HTML transferred: 77701092 bytes
Requests per second: 154.93 [#/sec] (mean)
Time per request: 1290.928 [ms] (mean)
Time per request: 6.455 [ms] (mean, across all concurrent requests)
Transfer rate: 5918.15 [Kbytes/sec] received

Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 242 1060.7 0 8999
Processing: 49 978 620.5 847 5457
Waiting: 47 948 588.9 838 5452
Total: 60 1220 1331.5 854 11031

Percentage of the requests served within a certain time (ms)
50% 854
66% 912
75% 978
80% 1041
90% 1888
95% 3859
98% 5641
99% 8201
100% 11031 (longest request)

Conclusion:

LiteSpeed:
Time per request: 26.056 [ms] (mean)
Time per request: 0.130 [ms] (mean, across all concurrent requests)

Varnish:
Time per request: 41.421 [ms] (mean)
Time per request: 0.207 [ms] (mean, across all concurrent requests)

Apache:
Time per request: 1290.928 [ms] (mean)
Time per request: 6.455 [ms] (mean, across all concurrent requests)

Benchmarks regarding a file of around 2GB will be next.

 

[IPMAN]

Let's Get Back To Work!

Hello,

Due to being very busy and a short period of ill health things regarding this testing came to a near stand-still, However am back and going to finish what I first started doing and that was showing the people of the internet the truths between Litespeed and varnish.

In our initial testing (Above) we used Unixy Varnish 1.3b so am unsure if theirs been any more stable releases since our initial testing however I shall look into this before starting anymore testing and will make sure everything has latest releases running before proceeding with any testing.

Lets Get Back Testing Folks!

 

[IPMAN]

Yes, Unixy I Did Use Your Plug-in!!!

Hello,

A thread post from WHT was brought to my attention by a close friend.

WHT Post: http://www.webhostingtalk.com/showpost.php?p=7378490&postcount=111

I am very baffled to think why Unixy assumes I didn’t use the Unixy Varnish Plug-in in my ab tests…

From Unixy himself “Whoever is running that show did not benchmark our plug-in.”

So I post this to set this straight, yes Unixy we did use your Varnish Plug-in and not the default VCL from the Varnish repository like your trying to make people believe.

Infact I used a trial version which was labelled: varnish.unixy.net-1.3b.tar

Why does Unixy feel the need to make up false half baked accusations?

I would also like to point out that yes a member of the Litespeed staff did install Litespeed for me and I also doubled checked to make sure it was a stock installation and nothing more!

I did notice that on the email which I received from Unixy it stated that their would be an installation charge for the Varnish product to be installed so in this instance I got one of my techs to install the Unixy Varnish Plug-in since I felt being charged for installation of a trial product was a little unfair.

So armed with the port numbers I was then myself able to run the simple ab tests as shown in my previous posts, All the ab tests was carried out by me!

For the people of WHT & Litespeed Forums I have enclosed some screenshots to actually prove that yes I did use a trial version of Unixy’s Varnish Plug-in and not the default VCL from the Varnish repository.