Non-parsed headers?

[satang]

Hi, I'm trying to use LiteSpeed to generate x-mixed-replace content via a FastCGI responder.

Using `strace(1)', I can see that my application is sending the response to LSWS and flushing completely. However, LSWS is not flushing the response to the client as the individual parts are received.

Note: the response I am sending to LSWS has been piped through "cat -vent" to show the control characters used.

     1  HTTP/1.0 200 OK^M$
     2  Expires: Tue, 29 Jun 2004 01:23:49 GMT^M$
     3  Date: Tue, 29 Jun 2004 01:33:49 GMT^M$
     4  Pragma: no-cache^M$
     5  Server: your mum (ports always open)^M$
     6  Content-Type: multipart/x-mixed-replace;boundary=OOK^M$
     7  ^M$
     8  ^M$
     9  --OOK^M$
    10  Content-Type: text/html^M$
    11  ^M$
    12  Hello, world!  This is response number 1$
    13  ^M$
    14  --OOK^M$
    15  Content-Type: multipart/x-mixed-replace;boundary=OOK^M$
    16  ^M$
    17  Hello, world!  This is response number 2$
    18  ^M$
    19  --OOK^M$
    20  Content-Type: text/html^M$
    21  ^M$
    22  Hello, world!  This is response number 3$
    23  ^M$
    24  --OOK^M$
    25  Content-Type: text/html^M$
    26  ^M$
    27  Goodbye, cruel world!$

[mistwang]

I think your fast CGI responder does not fully comply with fast CGI specification. LSWS buffers response until "FCGI_END_REQUEST" packet is received for such a small response body. I believe, LSWS does not receive "FCGI_END_REQUEST" in your case.

In order to help debuging the problem, you can turn on LSWS debug log by setting debug level to "HIGH" from the web admin interface.

What language is being used? Are you using any Fast CGI SDK or implementing the Fast CGI protocol stack yourself?

[satang]

Originally Posted by mistwang

I think your fast CGI responder does not fully comply with fast CGI specification. LSWS buffers response until "FCGI_END_REQUEST" packet is received for such a small response body. I believe, LSWS does not receive "FCGI_END_REQUEST" in your case.

In order to help debuging the problem, you can turn on LSWS debug log by setting debug level to "HIGH" from the web admin interface.

What language is being used? Are you using any Fast CGI SDK or implementing the Fast CGI protocol stack yourself?

Thanks for your response. As detailed in http://www.fastcgi.com/devkit/doc/fcgi-spec.html#S5.5, FCGI_END_REQUEST should only be used at the end of the response. As the response consists of multiple parts (with a second or two delay between each one), there is only one logical request to the FCGI communications layer.

Apache handles this by turning off buffering if the script issues a complete HTTP compliant header (including the response code line) - see http://httpd.apache.org/docs/misc/FA...ml#nph-scripts.

FYI, I'm using Perl's FCGI module.

2004-06-29 14:27:16.117 [DEBUG] [192.168.1.30:40428-1] Keep-alive timeout, close!
2004-06-29 14:27:16.117 [DEBUG] [192.168.1.30:40428-1] Close socket ...
2004-06-29 14:27:18.716 [DEBUG] [*:80] New connection from 192.168.1.30:40430.
2004-06-29 14:27:18.716 [DEBUG] [*:80] 1 connections accepted!
2004-06-29 14:27:21.875 [DEBUG] [192.168.1.30:40430-0] HttpIOLink::handleEvents() events=1!
2004-06-29 14:27:21.875 [DEBUG] [192.168.1.30:40430-0] HttpConnection::onReadEx()!
2004-06-29 14:27:21.875 [DEBUG] [192.168.1.30:40430-0] Read from client: 46
2004-06-29 14:27:21.875 [DEBUG] [192.168.1.30:40430-0] HttpIOLink::handleEvents() events=1!
2004-06-29 14:27:21.875 [DEBUG] [192.168.1.30:40430-0] HttpConnection::onReadEx()!
2004-06-29 14:27:21.875 [DEBUG] [192.168.1.30:40430-0] Read from client: 89
2004-06-29 14:27:21.876 [DEBUG] [192.168.1.30:40430-0] New request: 
        Method=[GET], URI=[/fozzie-dev/test/server-push.pl],
        QueryString=[]
        Content Length=0
2004-06-29 14:27:21.876 [DEBUG] [192.168.1.30:40430-0] HttpIOLink::suspendRead()...
2004-06-29 14:27:21.876 [DEBUG] [192.168.1.30:40430-0] run fcgi processor.
2004-06-29 14:27:21.876 [DEBUG] [UDS://mv/app/dev/fozzie/var/appSocket] connection available!
2004-06-29 14:27:21.876 [DEBUG] [UDS://mv/app/dev/fozzie/var/appSocket] new request [192.168.1.30:40430-0:fcgi] is assigned with connection!
2004-06-29 14:27:21.876 [DEBUG] [192.168.1.30:40430-0:fcgi] FcgiConnection::doWrite()
2004-06-29 14:27:21.876 [DEBUG] [192.168.1.30:40430-0:fcgi] FcgiConnection::beginRequest()
2004-06-29 14:27:21.876 [DEBUG] [192.168.1.30:40430-0:fcgi] FcgiConnection::pendingWrite(),m_iCurStreamHeader=16
2004-06-29 14:27:21.876 [DEBUG] [192.168.1.30:40430-0:fcgi] request header is done
2004-06-29 14:27:21.876 [DEBUG] [192.168.1.30:40430-0:fcgi] FcgiConnection::beginReqBody()
2004-06-29 14:27:21.876 [DEBUG] [192.168.1.30:40430-0:fcgi] FcgiConnection::pendingEndStream()
2004-06-29 14:27:21.876 [DEBUG] [192.168.1.30:40430-0:fcgi] ExtConn::continueRead()
2004-06-29 14:27:21.876 [DEBUG] [192.168.1.30:40430-0:fcgi] Request body done!
2004-06-29 14:27:21.876 [DEBUG] [192.168.1.30:40430-0:fcgi] FcgiConnection::endOfReqBody()
2004-06-29 14:27:21.876 [DEBUG] [192.168.1.30:40430-0:fcgi] FcgiConnection::pendingEndStream()
2004-06-29 14:27:21.876 [DEBUG] [192.168.1.30:40430-0:fcgi] FcgiConnection::flush()
2004-06-29 14:27:21.880 [DEBUG] [192.168.1.30:40430-0:fcgi] FcgiConnection::suspendWrite()

... here is where it sits until the last chunk is sent from the application ...

2004-06-29 14:27:27.919 [DEBUG] [192.168.1.30:40430-0:fcgi] ExtConn::onRead()
2004-06-29 14:27:27.919 [DEBUG] [192.168.1.30:40430-0:fcgi] FcgiConnection::doRead()
2004-06-29 14:27:27.919 [DEBUG] [192.168.1.30:40430-0:fcgi] read 560 bytes from Fast CGI.
2004-06-29 14:27:27.919 [DEBUG] [192.168.1.30:40430-0:fcgi] FCGI Header: 01060001020c0400
2004-06-29 14:27:27.919 [DEBUG] [192.168.1.30:40430-0:fcgi] process STDOUT 524 bytes
2004-06-29 14:27:27.919 [DEBUG] [192.168.1.30:40430-0:fcgi] response header finished!
2004-06-29 14:27:27.919 [DEBUG] [192.168.1.30:40430-0:fcgi] FCGI Header: 0106000100000000
2004-06-29 14:27:27.919 [DEBUG] [192.168.1.30:40430-0:fcgi] FCGI Header: 0103000100080000
2004-06-29 14:27:27.919 [DEBUG] [192.168.1.30:40430-0:fcgi] [EXT] EndResponse( endCode=0, protocolStatus=0 )
2004-06-29 14:27:27.919 [DEBUG] [192.168.1.30:40430-0:fcgi] call pConn->writeRespBody() to write 318 bytes
2004-06-29 14:27:27.919 [DEBUG] [192.168.1.30:40430-0:fcgi] Written to client: 548
2004-06-29 14:27:27.919 [DEBUG] [192.168.1.30:40430-0:fcgi] pConn->writeRespBody() return 318
2004-06-29 14:27:27.919 [DEBUG] [192.168.1.30:40430-0:fcgi] ReqBody: 0, RespBody: 318, HEC_COMPLETE!
2004-06-29 14:27:27.919 [DEBUG] [192.168.1.30:40430-0:fcgi] m_pHandler->onWrite() return 0
2004-06-29 14:27:27.919 [DEBUG] [192.168.1.30:40430-0:fcgi] HttpConnection::flush()!
2004-06-29 14:27:27.919 [DEBUG] [192.168.1.30:40430-0:fcgi] HttpConnection::nextRequest()!
2004-06-29 14:27:27.919 [DEBUG] [192.168.1.30:40430-0:fcgi] HttpExtConnector::cleanUp() ...
2004-06-29 14:27:27.919 [DEBUG] [192.168.1.30:40430-1] release ExtProcessor!
2004-06-29 14:27:27.919 [DEBUG] [UDS://mv/app/dev/fozzie/var/appSocket] add recycled connection to connection pool!
2004-06-29 14:27:32.144 [DEBUG] [192.168.1.30:40430-1] Keep-alive timeout, close!
2004-06-29 14:27:32.144 [DEBUG] [192.168.1.30:40430-1] Close socket ...

[mistwang]

I see.
The NPH feature will be supported in next release.

We buffer the response for the following reasons, to determine whether to compress the response, to maximize the compression ratio, to avoid TCP packet fragmentations, etc. And, if the response fragment is big enough, it will be sent to client immediately. In some cases, unbuffered might be preferred.

However, the log shows that the output is still buffered on the fast CGI side.

Thank you for the detail explanation. :-)
George

[satang]

Yes, there are lots of good reasons to buffer.

An alternative to going down the nph vs normal mode might be to simply use Nagle's algorithm (flush output buffer if it is full, or if there was N ms of inactivity). This is of course really simple to implement in userspace threaded / event based programs.

A tunable Nagle buffer would be the most flexible solution. Set it to 0ms for unbuffered behaviour, or to 100ms to avoid fragmentation. I guess this would be a setting for external applications. When the response is closed, the buffer is immediately flushed of course.

Then, using "nph" wouldn't be so fiddly. Deciding whether to buffer output based on whether a full HTTP header was output by the script is, after all, a horrid hack, typical of Apache! :-)

[mistwang]

Yes, it is simple to use Nagle's algorithm in some cases, however, it will degrade server performance if not being used wisely, we do not want to open another can of worm here. Thank you for the suggestion any way. :-)

It is trivial for us to implement the "nph" feature as every thing required is there already.

I agree with you that "nph" hack is not a good thing, instead, we can easily add a configuration entry to control the buffered/unbuffered behavior for any request, but it is not bad idea to has an Apache compatible mode.

[satang]

Originally Posted by mistwang

Yes, it is simple to use Nagle's algorithm in some cases, however, it will degrade server performance if not being used wisely,

Indeed. Which is why if you were to implement it, you'd only employ it in situations where the buffer would otherwise sit there and stagnate. Normal behaviour - serving files, and flushing responses once they have been completely received would proceed as normal.

[mistwang]

Please try release 1.5.5

[satang]

Originally Posted by mistwang

Please try release 1.5.5

Fantastic, it works as expected. For closure of the topic, here is my test program, which works well on my Perl 5.8.4 platform (FCGI module version = 0.67)

#!/usr/bin/perl

# Copyright (c) 2004, Sam Vilain.  This program is free software; you may
# use it and/or modify it under the same terms as Perl itself.

use strict;
use FCGI;
use IO::Handle;

# to demonstrate the effect of not sending a "full" header
my $use_nph = $ENV{USE_NPH} || 1;

# external application mode, UDS can be specified with FCGI_SOCKET
# environment variable
my $socket = $ENV{FCGI_SOCKET};
my $old_mask = umask 0;
my $sock = FCGI::OpenSocket($socket, 5) if $socket;
umask $old_mask;

my ($in, $out, $err) = map { IO::Handle->new() } (1..3);
my %env;
my $request = FCGI::Request($in, $out, $err, \%env,
			    $sock, &FCGI::FAIL_ACCEPT_ON_INTR);

while ( $request->Accept() >= 0 ) {

    select $out;

    print "HTTP/1.1 200 OK\r\n" if $use_nph;
    print( "Content-Type: multipart/x-mixed-replace;boundary=OOK\r\n",
	   "Pragma: no-cache\r\n\r\n"
	 );
    $request->Flush();

    for ( 1..3 ) {
	print "--OOK\r\nContent-Type: text/plain\r\n\r\n";
	print "Hello, server push - response number $_\n";
	$request->Flush();
	sleep 2;
	print "extra content!\n";
	$request->Flush();
	sleep 2;
    }
}