Re: making a file sparse - in-place?

[Pádraig Brady]

On 01/24/2014 03:56 AM, Rodrigo Campos wrote:
> On Fri, Jan 24, 2014 at 03:30:20AM +0000, Pádraig Brady wrote:
>> On 01/24/2014 03:12 AM, Rodrigo Campos wrote:
>>> On Fri, Jan 24, 2014 at 02:59:41AM +0000, Pádraig Brady wrote:
>>>> On 01/24/2014 02:41 AM, Rodrigo Campos wrote:
>>>>> On Fri, Jan 24, 2014 at 01:07:21AM +0000, Pádraig Brady wrote:
>>>>>> On 01/24/2014 12:47 AM, Bernhard Voelker wrote:
>>>>>>> Inspired by a recent post on util-linux ML [1], talking about turning
>>>>>>> a file into a sparse file in-place, i.e. not using a 2-step approach
>>>>>>> like `cp --sparse file file2 && mv file2 file`), I thought, hey, don't
>>>>>>> we have this in coreutils already?
>>>>>>
>>>
>>> But, a question about policy: is it okay to implement linux-only extensions 
>>> here ?
>>
>> If the current system doesn't support in place sparsify,
>> then be could document that limitation along the
>> same lines as the conv=notrunc case above.
> 
> Sorry, not sure I follow you here. If we have the same documentation, the
> conv=notrunc case never says anything platform nor filesystem specific. And 
> that
> is a big difference, IMHO. And, also, truncate() is a quite common function.

I meant there are cases where the current conv=sparse code
doesn't create a sparse file when the dest is opened with conv=notrunc.
We documented that, which is was I pasted here. Similarly we could
document caveats with in place sparsification.

> I mean, is there any linux specific extension to a tool in coreutils ?
> (optimization underneath, using linux-only features is totally different 
> IMHO).

In place punch hole may not work on all file systems anyway.
So the system specific point is moot here. We would always
have to handle cases where it was not supported.

> But it's just a matter of policy. Do you know what the policy is ?

We use system specific functionality where appropriate.
We would just decide what to do when the current
file doesn't support punch hole (which we have to do anyway as said above).
Do we fail or ignore. Generally sparse requests are advisory,
and indeed in dd we already quietly disable sparse mode if not supported.
So we would do the same if punch hole was not supported on the dest.

>> If one wanted more portable guarantees about sparsifying a file,
>> then it would be best to use a temporary file anyway.
> 
> ¿? What do you mean by this ?

Doing it that way would work in more situations

>> If there are other methods to punch a hole in a file
>> on other systems, they can be added as an option to coreutils
>> without changing the interface.
> 
> AFAIK, there aren't. And, as I said, is not linux specific only, it also is fs
> specific.

There is fcntl(F_FREESP) on solaris, with mention of porting that to FREEBSD.

> If I have to guess, this flag is like other useful flags linux adds and most
> other platforms don't. So you are exposing a feature here, that will only will
> work on linux with certain filesystems. Probably there is a policy already
> esablished for this ? (I mean, if it's ok or not)

If there is a sensible fall back, then it's OK to use non ubiquitous interfaces.

thanks,
Pádraig.