Re: [EXTERNAL] Re: New command consideration - curtail

[Pádraig Brady]

On 12/11/18 06:03, Wolaver, David wrote:
> On 08/11/18 09:02, Wolaver, David wrote:
>> Hi all,
>>
>> I have developed a command line program that I would like to have considered 
>> for inclusion in a future revision of coreutils.  The name of the program is 
>> "curtail".  It reads stdin and appends to an output file while ensuring that 
>> the file will never exceed a specified maximum size.  The main intent is to 
>> place a bound on the output of a program such that it cannot create a 
>> runaway log file.  The output file is truncated (in multiple of block size) 
>> at the beginning such that the output file always contains the latest output 
>> of the program.
>>
>> user_program | curtail -s 4M output.txt
> 
> It's a perfect name for such a command anyway :)
> 
> Does this depend on FALLOC_FL_COLLAPSE_RANGE support.
> Would this command fail on file systems not supporting this?
> Being block based seems a little problematic, especially considering 
> multi-byte characters, but maybe that's OK.
> 
> cheers,
> Pádraig
> 
> Hi Pádraig,
> 
> Thanks for spending some time to think about curtail.  I purposely omitted 
> implementation details from the first post because they are irrelevant unless 
> the following can be established...
> 
> 1. There exists problem(s) that can be addressed by such a program (I have 
> only presented one scenario)
> 2. Consensus that existing coreutils/scripts cannot produce the same result - 
> at least not in an efficient manner
> 3. A solution to the problem belongs in coreutils
> 
> To answer your question.. yes FALLOC_FL_COLLAPSE_RANGE is the mechanism by 
> which files are truncated in the Linux implementation.  Any OS/filesystem can 
> be supported if it provides a mechanism to query block size and remove blocks 
> from the beginning of a file.

Implementation details do consolidate things in my mind at least.
They're important as coreutils tend to be general in that
they'll fallback to more generic solutions where system support
for certain features is lacking.

As for the need for a separate command, playing devil's advocate,
given that this is not just a view of the data (like tail(1)),
and controls the eviction of older data, perhaps this functionality
is less generally useful, and more suited as an operation mode
of logrotate etc.

cheers,
Pádraig