Re: [PATCH] read-response-body should return received data when any break happens

[Daniel Hartwig]

On 11 March 2012 23:35, Nala Ginrut <address@hidden> wrote:
>
> The new read-response-body will add the received data to the exceptional
> information which used by "throw", if read-response-body can't continue to
> work anymore, the received data will return with throw.
> And there's a useful helper function to write this data to the disk ==>
> (output-received-response-body e port)
>

> Any comments?
>

So this is an interesting start.  The idea of buffering the transfer
is great -- however, it falls short in this implementation because it
is internal to read-response-body.

Also, the whole business about passing the partial data out via an
exception is very messy.

What about also passing a bytevector to read-response-body?  The
exception then only needs to mention how many bytes were read because
the caller already has access to the bytevector aka the data.

Consider this quick hack:

(define* (read-response-body! r bv #:optional
                              (start 0)
                              (count (min (bytevector-length bv)
                                          (response-content-length r))))
  (and count
       (let ((read (get-bytevector-n! (response-port r) bv start count)))
         (if (= read count)
             bv
             (bad-response
              "EOF while reading response body: ~a bytes of ~a (buffer)"
              read count)))))

which has all the features of your solution yet is much smaller and
puts the caller in more explicit control of the buffering, which opens
up many scenarios.

For example, reusing the same bytevector and looping over
read-response-body! saving the results to disk each time.  This limits
the memory use to the size of the bytevector *and* removes the copy
operation from your implementation (bonus!).

This also facilitates the ongoing work with filtering streams, etc.
that some other people are tackling.

Some food for thought.