[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: bug: dd copies a random number of bytes
From: |
Robb Matzke |
Subject: |
Re: bug: dd copies a random number of bytes |
Date: |
Mon, 23 Feb 2009 14:11:40 -0500 |
Ah, thanks. I get it now. The full and partial blocks do indeed always add
up to what I would expect. It might be good if this were explicit in the man
page--I was expecing dd to try harder to fill its buffer each time no matter
how large a value for "bs=N".
Thanks,
--Robb
On Mon, 23 Feb 2009 19:46:39 +0100
Jim Meyering <jim@meyering.net> wrote:
> Robb Matzke wrote:
> > I'm actually not concerned with the SIGPIPE and expect it. What concerns me
> > is that the final counts reported by *dd* are wrong. It should report 1024
> > records copied, but instead reports a seemingly random and variable amount.
> >
> > Why is dd closing its stdin earlier than I asked it to? Why is dd
> > transfering
> > a random amount of data?
>
> You chose a block size of 1MB.
> That is way larger than will ever be filled via a pipe,
> so every read gets a partial "block".
> dd stops after 1024 of those.
>
> The output you see,
>
> 0+1024 records in
> 0+1024 records out
> 4259840 bytes (4.3 MB) copied, 0.00729445 s, 584 MB/s
>
> tells you that there were 0 full blocks read and 1024 partials.
>
> ...
> >> > cat /dev/zero |dd bs=1MB count=1024 of=/dev/null
> >> > cat /dev/null |dd bs=1MB count=1024 >/dev/null
> >> >
> >> > An strace on dd shows that the last read/write pair both succeed and
> >> > then dd
> >> > inexplicably closes both files. None of the reads return zero or
> >> > failure.
> >> > An strace on cat shows that its last write to stdout gets a sigpipe, so
> >> > it
> >> > really is trying to send the data to dd and it's dd that's closing the
> >> > pipe.
> >>
> >> That's expected and in fact required.
> >> You're telling dd to exit after reading 1024*1MB.
> >> Once dd exits, it closes its side of the pipe, but cat is still writing
> >> to the other end. Writing to a closed pipe provokes a SIGPIPE, by default.
> >> How much cat actually writes before it's killed by the
> >> SIGPIPE depends on kernel buffering, hence the variability.
> >>
> >> > Replacing /dev/null with a real file exhibits the behavior one would
> >> > expect.
> >> > Substituting some other I/O-counting command for dd works fine.
--
Robb Matzke L-159 | Ph: 812-967-6325 | I/O, I/O, it's off to disk we go,
Lawrence Livermore | Fax: 812-967-6326 | A petabyte to read or write,
National Laboratory | Hrs: 3am-11am PDT | I/O, I/O.