bug#32528: http-post breaks with XML response payload containing boundary

[Mark H Weaver]

Ricardo Wurmus <address@hidden> writes:

> I’m having a problem with http-post and I think it might be a bug.  I’m
> talking to a Debbugs SOAP service over HTTP by sending (via POST) an XML
> request.  The Debbugs SOAP service responds with a string of XML.
>
> Here’s a simplified version of what I do:
>
>   (use-module (web http))
>   (let ((req-xml "<soap:Envelope xmlns:soap...>"))
>     (receive (response body)
>         (http-post uri
>                    #:body req-xml
>                    #:headers
>                    `((content-type . (text/xml))
>                      (content-length . ,(string-length req-xml))))
>      ;; Do something with the response body
>      (xml->sxml body #:trim-whitespace? #t)))
>
> This fails for some requests with an error like this:
>
>     web/http.scm:1609:23: Bad Content-Type header: multipart/related; 
> type="text/xml"; start="<main_envelope>"; boundary="=-=-="

[...]

> The reason why it fails is that Guile processes the response and treats
> the *payload* contained in the XML response as HTTP.

No, this was a good guess, but it's not actually the problem.

If you add --save-headers to the wget command line, you'll see the full
response, and the HTTP headers are what's being parsed, as it should be.
It looks like this (except that I removed the carriage returns below):

  HTTP/1.1 200 OK
  Date: Tue, 28 Aug 2018 21:40:30 GMT
  Server: Apache
  SOAPServer: SOAP::Lite/Perl/1.11
  Strict-Transport-Security: max-age=63072000
  Content-Length: 32650
  X-Content-Type-Options: nosniff
  X-Frame-Options: sameorigin
  X-XSS-Protection: 1; mode=block
  Keep-Alive: timeout=5, max=100
  Connection: Keep-Alive
  Content-Type: multipart/related; type="text/xml"; start="<main_envelope>"; 
boundary="=-=-="
  
  <?xml [...]

The problem is simply that our Content-Type header parser is broken.
It's very simplistic and merely splits the string wherever ';' is found,
and then checks to make sure there's only one '=' in each parameter,
without taking into account that quoted strings in the parameters might
include those characters.

I'll work on a proper parser for Content-Type headers.

      Thanks,
        Mark