Re: Strange hole creation behavior

[Eric Sandeen]

On 4/11/14, 5:58 PM, Pádraig Brady wrote:
> On 04/11/2014 09:43 PM, Brian Foster wrote:
>> On Fri, Apr 11, 2014 at 06:13:59PM +0100, Pádraig Brady wrote:
>>> So this coreutils test is failing on XFS:
>>> http://git.sv.gnu.org/gitweb/?p=coreutils.git;a=blob;f=tests/dd/sparse.sh;h=06efc7017
>>> Specifically the last hole check on line 66.
>>>
>>> In summary what's happening is that a write(1MiB), lseek(1MiB), write(1MiB)
>>> creates only a 64KiB hole. Is that expected?
>>>
>>
>> This is expected behavior due to speculative preallocation. An FAQ with
>> regard to this behavior is pending, but see here for reference:
>>
>> http://oss.sgi.com/archives/xfs/2014-04/msg00083.html
>>
>> In that particular write(1MB), lseek(+1MB), write(1MB) workload, each
>> write is preallocating some extra space beyond the current EOF. The seek
>> then moves past that space, but the space doesn't go away. The
>> subsequent writes will extend EOF. The previously preallocated space now
>> resides in the middle of the file and can't be trimmed away when the
>> file is closed.
>>
>>> Now a 1MiB hole is supported using truncate:
>>>   dd if=/dev/urandom of=file.in bs=1M count=1 iflag=fullblock
>>>   truncate -s+1M file.in
>>>   dd if=/dev/urandom of=file.in bs=1M count=1 iflag=fullblock conv=notrunc 
>>> oflag=append
>>>   $ du -k file.in
>>>   2048  file.in
>>>
>>
>> This works simply because it is broken into multiple commands. When the
>> first dd exits, the excess space is trimmed off (the file descriptor is
>> closed). The subsequent truncate extends the file size without any
>> extra space getting caught between the old and new EOF.
>>
>> You can confirm this by using the 'allocsize=4k' mount option to the XFS
>> mount. If you wanted something more generic for the purpose of testing
>> the coreutils functionality, you could also set the size of file.out in
>> advance. E.g., with preallocation in effect:
>>
>> # dd if=file.in of=file.out bs=1M conv=sparse
>> # xfs_bmap -v file.out 
>> file.out:
>>  EXT: FILE-OFFSET      BLOCK-RANGE      AG AG-OFFSET        TOTAL
>>    0: [0..3967]:       9773944..9777911  1 (9080..13047)     3968
>>    1: [3968..4095]:    hole                                   128
>>    2: [4096..6143]:    9778040..9780087  1 (13176..15223)    2048
>>
>> ... and then prevent preallocation by ensuring writes do not extend the
>> file:
>>
>> # rm -f file.out 
>> # truncate --size=3M file.out
>> # dd if=file.in of=file.out bs=1M conv=sparse,notrunc
>> # xfs_bmap -v file.out 
>> file.out:
>>  EXT: FILE-OFFSET      BLOCK-RANGE      AG AG-OFFSET        TOTAL
>>    0: [0..2047]:       9773944..9775991  1 (9080..11127)     2048
>>    1: [2048..4095]:    hole                                  2048
>>    2: [4096..6143]:    9778040..9780087  1 (13176..15223)    2048
>>
>> Hope that helps.
> 
> Excellent info thanks.
> With that I can adjust the test so it passes (patch attached).
> 
> So for reference this means that cp can no longer recreate holes
> <= 1MiB from source to dest (with the default XFS allocation size):

Well, the allocation size changes based on the filesize; there's a
heuristic involved.  So I fear that if you hard-code it into your
tests, you risk failing again in the future...

> We could I suppose use FALLOC_FL_PUNCH_HOLE where available
> to cater for this case. I'll see whether this is worth adding.

That might make sense.

But filesystems get to pick their layout; even ext4 may opportunistically
fill in holes, etc - so I think you need to be pretty careful with these
sorts of tests...

-Eric