Re: [GNUnet-developers] HTTP

[Christian Grothoff]

On Sonntag, 8. Juni 2003 14:49 you wrote:
> Hi,
>
> the attached patch adds (incomplete) support for HTTP through a proxy
> server.
> Unfortunately, the proxy server Squid (www.squid-cache.org) doesn't support
> chunked transfer-encoding and insists on getting a "Content-Length"-Field.
> That blows the current design of the HTTP-Transport.

As far as I can see, the patch is only for the http 1.0 download of the 
HOSTLISTURL and not the HTTP transport service, but that aspect is complete. 
It does not seem to break anything, so I've put it into CVS (didn't test it, 
though). 

As for the HTTP transport, with a proxy it would be difficult to *receive* 
connections anyway (would have to be uni-directional). Also, without the 
chunked encoding, we would have to re-establish connections again and again, 
which would be terrible. A better solution would be to set the content-length 
to a very large number (say 2 GB) and then just transmit chunks with their 
own length information just like the TCP transport. That might work, too. 

> Perhaps we need a separate connect(), POST and close() for each message we
> send.

Yuck. :-). 

> Another problem is, that peers behind a proxy can't accept connections from
> the outside. So the proxied peer has to maintain "GET"-connections to all
> the peers it is connected to in order to receive messages.

Not really, look at the code, it is bi-directional, just like TCP behind a 
NAT box. We don't do GET, we do PUSH with a 200 OK response -- and then we 
have both sides transmitting chunks. The only thing with a proxy (besides not 
supported chunked encoding) is that you can not be contacted, so you need to 
run another transport, such as SMTP, to receive inbound messages.

> Any comments or other suggestions?

I don't know if we really need proxy support at this stage (other than for 
the HOSTLISTURL, where it makes a lot of sense). If we do, we may have to 
reconsider the way the HTTP blocks are currently encapsulated. But doing 
permanent TCP reconnects is out of the question. I've recently bought a book 
on HTTP, so if I ever get to read it, I may have a better idea :-).

Do you see a reason why the "utopic content-size field" idea would not work? 
I didn't know the proxy problem and was just playing with HTTP when I put in 
the chunked code. We may have alternatives...
 
Christian