litespeed serving index.html.gz incorrectly (re-compressed?)

[aww]

On a new install where wp-super-cache is trying to serve it's own pre-compressed files, litespeed seems to be sending them as binary when apache gets it right.

Perhaps I need to exclude recompressing them somehow?

Works fine in apache, litepspeed serves garbled (looks like native gzip)

Headers are almost indentical.

litespeed

HTTP/1.1 200 OK
Content-Encoding: gzip
Vary: Accept-Encoding, Accept-Encoding, Cookie
Date: Fri, 23 Dec 2011 09:07:02 GMT
Server: LiteSpeed
Accept-Ranges: bytes
Connection: Keep-Alive
Keep-Alive: timeout=5, max=100
Etag: "2483-4ef43dba-0"
Last-Modified: Fri, 23 Dec 2011 08:37:14 GMT
Content-Type: text/html; charset=UTF-8
Content-Length: 9370
Cache-Control: max-age=3, must-revalidate

apache

HTTP/1.1 200 OK
Date: Fri, 23 Dec 2011 09:06:22 GMT
Last-Modified: Fri, 23 Dec 2011 08:37:14 GMT
Accept-Ranges: bytes
Content-Length: 9347
Cache-Control: max-age=3, must-revalidate
Expires: Fri, 23 Dec 2011 09:06:25 GMT
Vary: Accept-Encoding,Cookie
Keep-Alive: timeout=5, max=100
Connection: Keep-Alive
Content-Type: text/html; charset=UTF-8
Content-Encoding: gzip

So the content type is the same and the critical content-encoding is there.

But my guess is that litespeed sees text/html content-type and tries to re-encode it with gzip because the filetype (html.gz) is not excluded. Length is slightly longer which also hints at re-compression with just a repeated compression header.

But I do not know how to exclude by extension, litespeed only supports mime-type?

Is there any possible easy workaround?

I know one solution is just to let litespeed make the gzip in it's own area but client wants apache backwards compatibility.

 

[aww]

Some searching shows this is a common problem with anything that tries to pre-cache .gz (also drupal)

http://www.litespeedtech.com/support/forum/showthread.php?t=5268

http://www.litespeedtech.com/support/forum/showthread.php?t=2010

http://www.litespeedtech.com/support/forum/showthread.php?t=3734

http://www.litespeedtech.com/support/forum/showthread.php?t=2226

I cannot find a way to trick litespeed into ignoring .gz files regardless of content-type

This does NOT solve the problem

<FilesMatch "(\.html\.gz|\.json\.gz)$">
Header set Content-Encoding "gzip"
</FilesMatch>

I guess I will have to force super-cache to not pre-gzip

Perhaps in 4.2 you can add compression exclusions based on physical file extension (not url but source)

Or filematch compression disabling via htaccess, maybe setting an ENV flag that litespeed will obey?


BTW the reason why you do not want to turn off the gzip management in wp-super-cache, is that on a busy site there may be hundreds if not thousands of files that are compressed and have to be expired periodically. Then litespeed has to do double the effort in watching for expired files, when they could have been purged directly in the super-cache directories. I use a cron to expire them efficiently so it doesn't overwhelm the OS.

 

[aww]

Until we figure out another workaround, rather than disable super-cache gzip, I simply added one rule to the top of their .htaccess rules to skip the .gz version for litespeed

This might help someone in the future as it's an instant workaround

RewriteCond %{SERVER_SOFTWARE} !LiteSpeed
RewriteCond %{REQUEST_URI} !^.*[^/]$
RewriteCond %{REQUEST_URI} !^.*//.*$
RewriteCond %{REQUEST_METHOD} !POST
RewriteCond %{QUERY_STRING} !.*=.*
RewriteCond %{HTTP:Cookie} !^.*(comment_author_|wordpress_logged_in|wp-postpass_).*$
RewriteCond %{HTTP:X-Wap-Profile} !^[a-z0-9\"]+ [NC]
RewriteCond %{HTTP:Profile} !^[a-z0-9\"]+ [NC]
RewriteCond %{HTTP:Accept-Encoding} gzip
RewriteCond %{HTTPS} !on
RewriteCond %{DOCUMENT_ROOT}/wp-content/cache/supercache/%{HTTP_HOST}/$1/index.html.gz -f
RewriteRule ^(.*) "/wp-content/cache/supercache/%{HTTP_HOST}/$1/index.html.gz" [L]

RewriteCond %{REQUEST_URI} !^.*[^/]$
RewriteCond %{REQUEST_URI} !^.*//.*$
RewriteCond %{REQUEST_METHOD} !POST
RewriteCond %{QUERY_STRING} !.*=.*
RewriteCond %{HTTP:Cookie} !^.*(comment_author_|wordpress_logged_in|wp-postpass_).*$
RewriteCond %{HTTP:X-Wap-Profile} !^[a-z0-9\"]+ [NC]
RewriteCond %{HTTP:Profile} !^[a-z0-9\"]+ [NC]
RewriteCond %{HTTPS} !on
RewriteCond %{DOCUMENT_ROOT}/wp-content/cache/supercache/%{HTTP_HOST}/$1/index.html -f
RewriteRule ^(.*) "/wp-content/cache/supercache/%{HTTP_HOST}/$1/index.html" [L]

Note how it falls through to the non-compressed version which is tested working.

 

[NiteWave]

But my guess is that litespeed sees text/html content-type and tries to re-encode it with gzip because the filetype (html.gz) is not excluded.

not sure if this rewriterule will help:

RewriteRule \.html\.gz$ - [E=no-gzip:1]

put it just before super-cache rules.

 

[aww]

Wouldn't you have to test the physical file and not the url ?

Unless I am mistaken, that rule tests the url

super-cache is rewriting a regular url to the .html.gz

But the point is moot anyway, I tested by accessing a direct .html.gz and it doesn't work either

though I have to find that new setting in 4.x where .htaccess is not processed immediately - I vaguely remember that being added, so maybe it's not in effect

 

[NiteWave]

Unless I am mistaken, that rule tests the url

you're right.

then how about
RewriteRule ^(.*) "/wp-content/cache/supercache/%{HTTP_HOST}/$1/index.html.gz" [L]
===>
RewriteRule ^(.*) "/wp-content/cache/supercache/%{HTTP_HOST}/$1/index.html.gz" [L,E=no-gzip:1]

 

[aww]

Sad to report that idea doesn't work either, but it seems clever and plausible.

Why would litespeed ignore that environment directive?

Is it just too late at that point to trigger the override?

 

[NiteWave]

did some tests in lab.

following rules in .htaccess are needed to make it working:

<FilesMatch "(\.html\.gz|\.json\.gz)$">
Header set Content-Encoding "gzip"
</FilesMatch>

RewriteRule ^(.*) "/wp-content/cache/supercache/%{HTTP_HOST}/$1/index.html.gz" [L]
===>
RewriteRule ^(.*) "/wp-content/cache/supercache/%{HTTP_HOST}/$1/index.html.gz" [L,E=no-gzip:1,T=text/html]

 

[aww]

Nice! Thank you very much for working it out.

I will apply it later today and confirm how it works live.

I did not know you could force mime types like that, apparently it really is in the mod-rewrite spec and litespeed implements it too. Fantastic work.

 

[mistwang]

This issue will be addressed in 4.1.10, it should work with WP super cache original configuration with gzip compression enabled.
Stay tuned.

 

[aww]

Wow even better. Top notch responsiveness from litespeed. Greatly appreciated.

Happy to beta test once a download is available.

 

[mistwang]

initial 4.1.10 build has uploaded, you can change version number in the download link to give it a try.

 

[Mr.Check]

I have a very similar problem with WordPress and Litespeed Web Server Enterprise v4.1.13:

I'm also using WP-Super-Cache that creates index.html.gz files for every web page. Sometimes I can see the website without any problems, and sometimes the browser shows me the gz-file. I used the web-sniffer.net service to to find out what exactly happens and causes the strange behavior.

I have figured out that sometimes LiteSpeed sends "Content-Encoding: gzip" with the HTTP header, and sometimes it does not. When it does not send this, the browser doesn't know that it has to decompress the gzip-content and shows the plain gzip-file. So I suspect that LiteSpeed does not always recognize the gzip-content and therefore it does not add the content-encoding to the HTTP header.

It does not matter if I enable or disable gzip-compression in LiteSpeed configuration. It's just that sometimes LiteSpeed delivers the gz-file from WP-Cache with "Content-Encoding: gzip" information, and sometimes it sends the gz-file without this important piece of information.

Any ideas what could be the reason for this problem?

 

[NiteWave]

is it easy or difficult to reproduce the issue?

 

[Mr.Check]

is it easy or difficult to reproduce the issue?

Hi NiteWave, thanks for your reply. On my server it is very easy to reproduce it. If you want me to, I can send to a PM with an URL you can test at web-sniffer.net and you'll see that sometimes LiteSpeed sends "Content-Encoding: gzip", and sometimes it does not.

 

[NiteWave]

OK. please PM. can monitor it for 1 or 2 days.