litespeed serving index.html.gz incorrectly (re-compressed?)

Discussion in 'Install/Configuration' started by aww, Dec 23, 2011.

  1. aww

    aww New Member

    On a new install where wp-super-cache is trying to serve it's own pre-compressed files, litespeed seems to be sending them as binary when apache gets it right.

    Perhaps I need to exclude recompressing them somehow?

    Works fine in apache, litepspeed serves garbled (looks like native gzip)

    Headers are almost indentical.

    litespeed
    apache

    So the content type is the same and the critical content-encoding is there.

    But my guess is that litespeed sees text/html content-type and tries to re-encode it with gzip because the filetype (html.gz) is not excluded. Length is slightly longer which also hints at re-compression with just a repeated compression header.

    But I do not know how to exclude by extension, litespeed only supports mime-type?

    Is there any possible easy workaround?

    I know one solution is just to let litespeed make the gzip in it's own area but client wants apache backwards compatibility.
    Last edited: Dec 23, 2011
  2. aww

    aww New Member

    Some searching shows this is a common problem with anything that tries to pre-cache .gz (also drupal)

    http://www.litespeedtech.com/support/forum/showthread.php?t=5268

    http://www.litespeedtech.com/support/forum/showthread.php?t=2010

    http://www.litespeedtech.com/support/forum/showthread.php?t=3734

    http://www.litespeedtech.com/support/forum/showthread.php?t=2226

    I cannot find a way to trick litespeed into ignoring .gz files regardless of content-type

    This does NOT solve the problem
    Code:
    <FilesMatch "(\.html\.gz|\.json\.gz)$">
    Header set Content-Encoding "gzip"
    </FilesMatch>
    
    I guess I will have to force super-cache to not pre-gzip

    Perhaps in 4.2 you can add compression exclusions based on physical file extension (not url but source)

    Or filematch compression disabling via htaccess, maybe setting an ENV flag that litespeed will obey?


    BTW the reason why you do not want to turn off the gzip management in wp-super-cache, is that on a busy site there may be hundreds if not thousands of files that are compressed and have to be expired periodically. Then litespeed has to do double the effort in watching for expired files, when they could have been purged directly in the super-cache directories. I use a cron to expire them efficiently so it doesn't overwhelm the OS.
    Last edited: Dec 23, 2011
  3. aww

    aww New Member

    Until we figure out another workaround, rather than disable super-cache gzip, I simply added one rule to the top of their .htaccess rules to skip the .gz version for litespeed

    This might help someone in the future as it's an instant workaround

    Code:
    RewriteCond %{SERVER_SOFTWARE} !LiteSpeed
    RewriteCond %{REQUEST_URI} !^.*[^/]$
    RewriteCond %{REQUEST_URI} !^.*//.*$
    RewriteCond %{REQUEST_METHOD} !POST
    RewriteCond %{QUERY_STRING} !.*=.*
    RewriteCond %{HTTP:Cookie} !^.*(comment_author_|wordpress_logged_in|wp-postpass_).*$
    RewriteCond %{HTTP:X-Wap-Profile} !^[a-z0-9\"]+ [NC]
    RewriteCond %{HTTP:Profile} !^[a-z0-9\"]+ [NC]
    RewriteCond %{HTTP:Accept-Encoding} gzip
    RewriteCond %{HTTPS} !on
    RewriteCond %{DOCUMENT_ROOT}/wp-content/cache/supercache/%{HTTP_HOST}/$1/index.html.gz -f
    RewriteRule ^(.*) "/wp-content/cache/supercache/%{HTTP_HOST}/$1/index.html.gz" [L]
    
    RewriteCond %{REQUEST_URI} !^.*[^/]$
    RewriteCond %{REQUEST_URI} !^.*//.*$
    RewriteCond %{REQUEST_METHOD} !POST
    RewriteCond %{QUERY_STRING} !.*=.*
    RewriteCond %{HTTP:Cookie} !^.*(comment_author_|wordpress_logged_in|wp-postpass_).*$
    RewriteCond %{HTTP:X-Wap-Profile} !^[a-z0-9\"]+ [NC]
    RewriteCond %{HTTP:Profile} !^[a-z0-9\"]+ [NC]
    RewriteCond %{HTTPS} !on
    RewriteCond %{DOCUMENT_ROOT}/wp-content/cache/supercache/%{HTTP_HOST}/$1/index.html -f
    RewriteRule ^(.*) "/wp-content/cache/supercache/%{HTTP_HOST}/$1/index.html" [L]
    
    Note how it falls through to the non-compressed version which is tested working.
  4. NiteWave

    NiteWave Administrator

    not sure if this rewriterule will help:

    RewriteRule \.html\.gz$ - [E=no-gzip:1]

    put it just before super-cache rules.
  5. aww

    aww New Member

    Wouldn't you have to test the physical file and not the url ?

    Unless I am mistaken, that rule tests the url

    super-cache is rewriting a regular url to the .html.gz

    But the point is moot anyway, I tested by accessing a direct .html.gz and it doesn't work either

    though I have to find that new setting in 4.x where .htaccess is not processed immediately - I vaguely remember that being added, so maybe it's not in effect
  6. NiteWave

    NiteWave Administrator

    you're right.

    then how about
    RewriteRule ^(.*) "/wp-content/cache/supercache/%{HTTP_HOST}/$1/index.html.gz" [L]
    ===>
    RewriteRule ^(.*) "/wp-content/cache/supercache/%{HTTP_HOST}/$1/index.html.gz" [L,E=no-gzip:1]
  7. aww

    aww New Member

    Sad to report that idea doesn't work either, but it seems clever and plausible.

    Why would litespeed ignore that environment directive?

    Is it just too late at that point to trigger the override?
  8. NiteWave

    NiteWave Administrator

    did some tests in lab.

    following rules in .htaccess are needed to make it working:

    <FilesMatch "(\.html\.gz|\.json\.gz)$">
    Header set Content-Encoding "gzip"
    </FilesMatch>

    RewriteRule ^(.*) "/wp-content/cache/supercache/%{HTTP_HOST}/$1/index.html.gz" [L]
    ===>
    RewriteRule ^(.*) "/wp-content/cache/supercache/%{HTTP_HOST}/$1/index.html.gz" [L,E=no-gzip:1,T=text/html]
  9. aww

    aww New Member

    Nice! Thank you very much for working it out.

    I will apply it later today and confirm how it works live.

    I did not know you could force mime types like that, apparently it really is in the mod-rewrite spec and litespeed implements it too. Fantastic work.
  10. mistwang

    mistwang LiteSpeed Staff

    This issue will be addressed in 4.1.10, it should work with WP super cache original configuration with gzip compression enabled.
    Stay tuned.
  11. aww

    aww New Member

    Wow even better. Top notch responsiveness from litespeed. Greatly appreciated.

    Happy to beta test once a download is available.
  12. mistwang

    mistwang LiteSpeed Staff

    initial 4.1.10 build has uploaded, you can change version number in the download link to give it a try.
  13. Mr.Check

    Mr.Check New Member

    I have a very similar problem with WordPress and Litespeed Web Server Enterprise v4.1.13:

    I'm also using WP-Super-Cache that creates index.html.gz files for every web page. Sometimes I can see the website without any problems, and sometimes the browser shows me the gz-file. I used the web-sniffer.net service to to find out what exactly happens and causes the strange behavior.

    I have figured out that sometimes LiteSpeed sends "Content-Encoding: gzip" with the HTTP header, and sometimes it does not. When it does not send this, the browser doesn't know that it has to decompress the gzip-content and shows the plain gzip-file. So I suspect that LiteSpeed does not always recognize the gzip-content and therefore it does not add the content-encoding to the HTTP header.

    It does not matter if I enable or disable gzip-compression in LiteSpeed configuration. It's just that sometimes LiteSpeed delivers the gz-file from WP-Cache with "Content-Encoding: gzip" information, and sometimes it sends the gz-file without this important piece of information.

    Any ideas what could be the reason for this problem?
  14. NiteWave

    NiteWave Administrator

    is it easy or difficult to reproduce the issue?
  15. Mr.Check

    Mr.Check New Member

    Hi NiteWave, thanks for your reply. On my server it is very easy to reproduce it. If you want me to, I can send to a PM with an URL you can test at web-sniffer.net and you'll see that sometimes LiteSpeed sends "Content-Encoding: gzip", and sometimes it does not.
  16. NiteWave

    NiteWave Administrator

    OK. please PM. can monitor it for 1 or 2 days.

Share This Page