LiteSpeed not returning Content-Encoding header for Range requests, breaks gzip

Discussion in 'Bug Reports' started by Sean Kinsey, Aug 1, 2014.

  1. Sean Kinsey

    Sean Kinsey New Member

    When combining `Accept-Encoding: deflate, gzip` and `Range: bytes=0-524281` as request headers, the response, both gzipped and limited to the range, is returned with a missing `Content-Encoding` header.
    This causes Curl and other clients to fail to decode the response.

    Code:
    oyvind-mbp:~ oyvind$ curl -v -A "facebookexternalhit/1.1 (+http://www.facebook.com/externalhit_uatext.php)" -H "Range: bytes=0-524281" --compressed "http://modernpracticality.com/"
    * Adding handle: conn: 0x7fe42c003a00
    * Adding handle: send: 0
    * Adding handle: recv: 0
    * Curl_addHandleToPipeline: length: 1
    * - Conn 0 (0x7fe42c003a00) send_pipe: 1, recv_pipe: 0
    * About to connect() to modernpracticality.com port 80 (#0)
    *   Trying 173.248.188.234...
    * Connected to modernpracticality.com (173.248.188.234) port 80 (#0)
    > GET / HTTP/1.1
    > User-Agent: facebookexternalhit/1.1 (+http://www.facebook.com/externalhit_uatext.php)
    > Host: modernpracticality.com
    > Accept: */*
    > Accept-Encoding: deflate, gzip
    > Range: bytes=0-524281
    >
    < HTTP/1.1 206 Partial Content
    < Date: Fri, 01 Aug 2014 04:50:32 GMT
    * Server LiteSpeed is not blacklisted
    < Server: LiteSpeed
    < Accept-Ranges: bytes
    < Connection: Keep-Alive
    < Keep-Alive: timeout=5, max=100
    < Last-Modified: Fri, 01 Aug 2014 04:48:10 GMT
    < Content-Type: text/html
    < X-Pingback: http://modernpracticality.com/xmlrpc.php
    < Vary: Accept-Encoding, Cookie
    < Content-Range: bytes 0-26570/26571
    < Content-Length: 26571
    We have seen this with multiple LiteSpeed powered domains, and so believe this is related to LiteSpeed, or how it interacts with other proxy components.

    (referenced from https://developers.facebook.com/bugs/335400146618351/)
  2. mistwang

    mistwang LiteSpeed Staff

    For Range request handled by LiteSpeed Enterprise Web Server itself, gzip compression is turned off, in other words, LiteSpeed does not apply gzip compressed to range requests.

    If it was compressed, it must be done by something else.

    When I try your test case, the page is served by PHP directly, without responding to the range request.
  3. Sean Kinsey

    Sean Kinsey New Member

    Hey @mistwang, although you might be right about LiteSpeed being self-consistent, this is an issue that we are only seeing with LiteSpeed servers. Attached is another sample:

    $ curl -v --compressed -H 'Connection: close' -H 'Range: bytes=0-524288' -A "facebookexternalhit/1.1 (+http://www.facebook.com/externalhit_uatext.php)" "http://www.republica.com.uy/" | head
    * Hostname was NOT found in DNS cache
    % Total % Received % Xferd Average Speed Time Time Time Current
    Dload Upload Total Spent Left Speed
    0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0* Trying 72.52.244.255...
    * Connected to www.republica.com.uy (72.52.244.255) port 80 (#0)
    > GET / HTTP/1.1
    > User-Agent: facebookexternalhit/1.1 (+http://www.facebook.com/externalhit_uatext.php)
    > Host: www.republica.com.uy
    > Accept: */*
    > Accept-Encoding: deflate, gzip
    > Connection: close
    > Range: bytes=0-524288
    >
    < HTTP/1.1 206 Partial Content
    < Date: Tue, 18 Nov 2014 18:10:02 GMT
    * Server LiteSpeed is not blacklisted
    < Server: LiteSpeed
    < Accept-Ranges: bytes
    < Connection: close
    < Last-Modified: Tue, 18 Nov 2014 18:09:54 GMT
    < Content-Type: text/html; charset=UTF-8
    < X-Pingback: http://www.republica.com.uy/xmlrpc.php
    < Vary: Accept-Encoding, Cookie
    < Pragma: public
    < Cache-Control: public
    < Content-Range: bytes 0-22802/22803
    < Content-Length: 22803
    <
    { [data not shown]
    �Ks#G�.���!�Hv!�"Y�G�:,�%���bY��th����D&���j��Y�屻��ؽf��5�B�1�z3f�2�����H `���f��Ȍ�G�����?���f�����=q���mQ���?/n��;G;�/_��j�"�B�Gn�������Qh�qg�\>;;+�-���Y>z[>G[UT6�V��Yrb���y�onT��J:��=��(T�F!jal'�p��/�V��B�I�k˒�KI�|ֱ�d���T[EeOg��k��b�!O�\��)�r�;Ǐ��b���V.SY_�q�!mU��%_Ž�\j��Ԡ��R���7Xe;��lȶ�u7<�mH7|��Fz^[�T���ʜ@�sQ��T�R*�q��6�bu������:��'�4���3��ucI��_����k�>X�[���P�|֍�i��,�T�0
  4. mistwang

    mistwang LiteSpeed Staff

    Please try the latest 4.2.19 build, we fixed a gzip related issue yesterday.
    If it does not fix it, we may need to login and take a look at it.
  5. Sean Kinsey

    Sean Kinsey New Member

    Hi @mistwang, I'm actually reporting this on behalf of Facebook, so won't be able to upgrade any implementations.
    I will refer bug reporters to this thread though.

    Would you be able to have someone verify if the bug could present itself as described in this thread?
  6. mistwang

    mistwang LiteSpeed Staff

    Sure, we will work with our client to figure it out.
    I probably know what is problem. Need to find a solution.

    Will facebook side takes a "200" response instead of "206 Partial Content" response contains the whole file?
  7. Sean Kinsey

    Sean Kinsey New Member

    Great, and yes, we'll accept a 200 response for the Range request.
  8. mistwang

    mistwang LiteSpeed Staff

    This issue should have been addressed in our latest 4.2.19 build. Any user reporting this issue should upgrade to 4.2.19 with

    /usr/local/lsws/admin/misc/lsup.sh -f -v 4.2.19
  9. brightestspark

    brightestspark New Member

    My host (VentraIP) is running Litespeed Server API 6.7. I have several wordpress sites on this server. They all have the Wordfence plugin. It has a gzip caching system called Falcon Engine. When that is enabled, I get garbled output on Facebook scrapes. Attached is what Facebook sees after a fresh scrape of one of my websites. Compress content is disabled in cPanel. I have no other caching/gzip functions. Wordfence keep telling me that my content is getting gzipped twice, which it is not.
    Refer https://wordpress.org/support/topic...ine-gzip-on-litespeed?replies=11#post-6272195

    Attached Files:

  10. brightestspark

    brightestspark New Member

    I just found out from my host that they are running Litespeed 4.2.15. I see this issue has been fixed in 4.2.19 so I will wait until they upgrade.

Share This Page