[Bug?] Browser caching. Missing Etag on "304 Not Modified".

#1
The problem:

With cache control is setup in .htaccess and when a "304 Not modified" response header status code is returned (through surfing between web pages), an Etag directive should be included in the response header. But it isn't: the Etag directive is missing.

This is a problem with respect to using Litespeed to effect the behaviour (through response headers) of a browser's cache. That is, it is not a problem of caching on the (Litespeed) server itself.

Steps to reproduce the problem:

[My production site www.philorum.org currently set up with the relevant .htaccess cache control for http://www.philorum.org/index.html. But I provide all the relevant steps to reproduce if you visit this thread in the distant future.]
  1. htaccess with cache-control set to: always validate browser cached version against server representation.

    # --------------------------------------------------
    # Etag Header
    # --------------------------------------------------
    # Include here as some Web Hosts have turned off Etags by default.
    # Don't use INode as part of the Etag as that will falsely flag a changed document
    # when a file is served from any of several servers (in a cluster)
    # http://httpd.apache.org/docs/2.4/mod/core.html#fileetag
    FileETag -INode +Size +MTime

    # --------------------------------------------------
    # Browser Caching Rule
    # --------------------------------------------------
    # Files (here just the home page, index) ought always be checked with the
    # server to see if the local copy is different from the server copy.
    # That is, always validate.
    <FilesMatch "^(index)\..+$">
    Header set Cache-Control "max-age=0, public"
    </FilesMatch>
  2. In Firefox ...
  3. Surf to http://www.philorum.org/index.html. Press F12 for developer tools > Network Tab. Click "reload" in developer tools Network tab.
  4. Ctrl + F5 for a forced retrieval of files from the server. In Developer Tools > Network > Click on the index.html record > Headers [Tab]
  5. Observe Status Code "200 OK" and, in the reponse header, (something like) Etag: ""19ac-5618a670-0"". This all looks good so far.
  6. On the webpage http://www.philorum.org/index.html click on "Draft Constitution" link, to surf away from index.html.
  7. Click on "Home" to surf back http://www.philorum.org/index.html . Observe, in Developer Tools > Network > Index.html > Headers .... Status Code "304 Not Modified" and, in the response header, no Etag.

Expected:

  • 7. Status Code "304 Not Modified" and, in the response header, an Etag (with a value identical to the previously cached Etag, to assert that index.html has not been modified).

Notes:

  • This is identified at https://redbot.org/?uri=http://www.philorum.org when mousing over the warnings for "The If-None-Match response is missing required headers" and "The If-Modified-Since response is missing required headers" there is information about the missing Etag.
  • This problem occurs whether the web pages are compressed (gzip) or not compressed.
  • In firefox (at least) a "304 not modified" response correctly does not come with a message payload (the full webpage) - it is taken from the cache. This is ultimately what the "304 not modified" response status code is for ... so the caching *is* behaving as expected and desired - however without the Etag as is required. (So the caching works as desired but by luck rather than design).
  • I'm new to Litespeed and don't have a local development copy. The site above, www.philorum.org, is served from a third party commercial web host which uses Litespeed (as is evidenced in the response headers). In short, it's entirely possible I'm missing something basic.

Edit: Simplified "Expected" description.
 
Last edited:

NiteWave

Administrator
#2
Sorry for reply late, read it once but forgot to reply.

let's do some tests in real world web page first.
1. apache
#curl -I http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html -H 'If-None-Match: "84d1-3e3073913b100"'
HTTP/1.1 304 Not Modified
Date: Tue, 03 Nov 2015 04:54:11 GMT
Server: Apache/2
ETag: "84d1-3e3073913b100"
Expires: Tue, 03 Nov 2015 10:54:11 GMT
Cache-Control: max-age=21600

2. nginx
#curl -I nginx.org -H 'If-None-Match: "563365f8-1e8b"'
HTTP/1.1 304 Not Modified
Server: nginx/1.7.7
Date: Tue, 03 Nov 2015 04:55:46 GMT
Last-Modified: Fri, 30 Oct 2015 12:43:36 GMT
Connection: keep-alive
Keep-Alive: timeout=15
ETag: "563365f8-1e8b"

3. litespeed
#curl -I http://blog.litespeedtech.com/wp-content/themes/litespeed/style.css -H 'If-None-Match: "2911-557745f3-d706c19f067a3b00"'
HTTP/1.1 304 Not Modified
Cache-Control: public, max-age=604800
Expires: Tue, 10 Nov 2015 04:56:36 GMT
Date: Tue, 03 Nov 2015 04:56:36 GMT
Accept-Ranges: bytes
Server: LiteSpeed

4. google web server
#curl -I www.google.com/images/nav_logo242.png -H 'if-modified-since: Thu, 22 Oct 2015 17:33:49 GMT'
HTTP/1.1 304 Not Modified
Date: Tue, 03 Nov 2015 04:57:23 GMT
Expires: Tue, 03 Nov 2015 04:57:23 GMT
Cache-Control: private, max-age=31536000
Last-Modified: Thu, 22 Oct 2015 17:33:49 GMT
X-Content-Type-Options: nosniff
Server: sffe
X-XSS-Protection: 1; mode=block

from above tests.
1. apache / nginx have "ETag" in response header if there is a request header "If-None-Match"
2. litespeed has no ETag in response header
3. tried a some static files from google. looks like google web server never generate ETag. instead, it'll generate
"Last-Modified" response header. and when response with "304 Not Modified", with "Last-Modified" in its response header when request header has proper 'if-modified-since" in it.

I'm not sure if this is a "bug" or if it's against RFC,
here's the official protocol document:
http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html
10.3.5 304 Not Modified

not easy to understand it well at the moment.
 

mistwang

LiteSpeed Staff
#3
This is caused by bug introduced in LSWS 5.0, it will be addressed in next build.
Anyway, browser today can tolerant that, it does not affect caching function.
 
#4
Thanks Nitewave, that was a helpful analysis. As you observed, your litespeed request reproduces the same result that I got (no e-tag on 304 not modified).

The google result you've turned up is interesting. Possibly they are still using the old school way of doing things, Etags for cache-control (edit: in addition to Last-Modifed) being new school.

Caching rules in the standards are tricky.

https://tools.ietf.org/html/rfc7232#section-2.4 might be clearer than the RFC you pointed to. For 200 OK responses:

In other words, the preferred behavior for an origin server is to send both a strong entity-tag [an E-Tag] and a Last-Modified value in successful responses to a retrieval request.
For 304 not modified responses, https://tools.ietf.org/html/rfc7232#section-4.1,

The server generating a 304 response MUST generate any of the following header fields that would have been sent in a 200 (OK) response to the same request: Cache-Control, Content-Location, Date, ETag, Expires, and Vary.
In reduced terms I understand this to mean: the preferred cache control response header is to include an Etag for 200 OK and 304 Not modified, whatever other response directives might be used.

And thank you mistwang, that confirms both: my suspicion (it was a bug); and my observation that caching function still works anyway.

Excellent result. Thanks folks.
 
Last edited:
Top