Non-parsed headers?
 

Hi, I'm trying to use LiteSpeed to generate x-mixed-replace content via a FastCGI responder.

Using `strace(1)', I can see that my application is sending the response to LSWS and flushing completely. However, LSWS is not flushing the response to the client as the individual parts are received.

Note: the response I am sending to LSWS has been piped through "cat -vent" to show the control characters used.

I think your fast CGI responder does not fully comply with fast CGI specification. LSWS buffers response until "FCGI_END_REQUEST" packet is received for such a small response body. I believe, LSWS does not receive "FCGI_END_REQUEST" in your case.

In order to help debuging the problem, you can turn on LSWS debug log by setting debug level to "HIGH" from the web admin interface.

What language is being used? Are you using any Fast CGI SDK or implementing the Fast CGI protocol stack yourself?

Thanks for your response. As detailed in http://www.fastcgi.com/devkit/doc/fcgi-spec.html#S5.5, FCGI_END_REQUEST should only be used at the end of the response. As the response consists of multiple parts (with a second or two delay between each one), there is only one logical request to the FCGI communications layer.

Apache handles this by turning off buffering if the script issues a complete HTTP compliant header (including the response code line) - see http://httpd.apache.org/docs/misc/FA...ml#nph-scripts.

FYI, I'm using Perl's FCGI module.

... here is where it sits until the last chunk is sent from the application ...

I see.
The NPH feature will be supported in next release.

We buffer the response for the following reasons, to determine whether to compress the response, to maximize the compression ratio, to avoid TCP packet fragmentations, etc. And, if the response fragment is big enough, it will be sent to client immediately. In some cases, unbuffered might be preferred.

However, the log shows that the output is still buffered on the fast CGI side.

Thank you for the detail explanation. :-)
George

Yes, there are lots of good reasons to buffer.

An alternative to going down the nph vs normal mode might be to simply use Nagle's algorithm (flush output buffer if it is full, or if there was N ms of inactivity). This is of course really simple to implement in userspace threaded / event based programs.

A tunable Nagle buffer would be the most flexible solution. Set it to 0ms for unbuffered behaviour, or to 100ms to avoid fragmentation. I guess this would be a setting for external applications. When the response is closed, the buffer is immediately flushed of course.

Then, using "nph" wouldn't be so fiddly. Deciding whether to buffer output based on whether a full HTTP header was output by the script is, after all, a horrid hack, typical of Apache! :-)

Yes, it is simple to use Nagle's algorithm in some cases, however, it will degrade server performance if not being used wisely, we do not want to open another can of worm here. Thank you for the suggestion any way. :-)

It is trivial for us to implement the "nph" feature as every thing required is there already.

I agree with you that "nph" hack is not a good thing, instead, we can easily add a configuration entry to control the buffered/unbuffered behavior for any request, but it is not bad idea to has an Apache compatible mode.

Indeed. Which is why if you were to implement it, you'd only employ it in situations where the buffer would otherwise sit there and stagnate. Normal behaviour - serving files, and flushing responses once they have been completely received would proceed as normal.

Please try release 1.5.5

Excellent! Here's a test program.
 

Fantastic, it works as expected. For closure of the topic, here is my test program, which works well on my Perl 5.8.4 platform (FCGI module version = 0.67)