Re: RFE: head,tail: -z, --zero-terminated

[Stephane Chazelas]

2015-10-19 22:47:24 -0400, Aaron Davies:
[...]
> here's an example of something i had to do a few months back:
> given a collection of directories, find the most-recently
> modified file in each directory and pass the list of those
> files to some utility as command-line arguments
> 
> here's the best i could come up with using mostly the standard
> shell tools while still properly supporting fully-arbitrary
> filenames (i don't think the sed on the system i was working
> on at the time includes the -z option)
> 
> (note the scripting uses some non-bourne features (arrays and
> function-local variables))

There's nothing standard in that unless by standard you mean the
tools present by default in most traditional general-purpose
(desktop/server) GNU/Linux distributions.

> the directories are a, b, and c; the utility is foo
> 
> function lastl { local r; while IFS= read -rd '' x; do r=$x; done; printf 
> '%s' "$r"; }
> unset files
> typeset -a files
> dirs=(a b c)
> while IFS= read -rd '' line
> do
>     files["${#files[@]}"]=$line
> done < <(
>     for dir in "${dirs[@]}"
>     do
>         find "$dir" -type f -printf '%T@\t%p\0' | sort -zk1,1n | lastl | perl 
> -0777ne 'printf("%s\0",$_)' | while IFS= read -rd '' l
>         do
>             tok0="${l%%$'\t*'}"
>             printf '%s\0' "${l:$((${#tok0}+1))}"
>         done
>     done
> )
> foo "${files[@]}"

That can be simplified greatly:

for dir in "${dirs[@]}"; do
  IFS=: read -rd '' discard "files[${#files[@]}]" < <(
    find "$dir" -type f -printf '%T@:%p\0' | sort -rzn)
done

(still fails for values of $dirs that start with - or are
find predicates)

Or since you're already invoking perl, you could do the thing
more easily with perl.

Or with zsh:

for dir ($dirs) files+=($dir/**/*(DN.om[1]))

Note that as already mentioned, you can always use the tr '\n\0'
'\0\n' trick (https://unix.stackexchange.com/a/75206).

That's a bit of a FAQ. See for instance:
https://unix.stackexchange.com/a/63750
https://unix.stackexchange.com/a/83430
https://unix.stackexchange.com/a/84137
https://unix.stackexchange.com/a/111693
https://unix.stackexchange.com/a/111822
https://unix.stackexchange.com/a/113843
https://unix.stackexchange.com/a/148893
https://unix.stackexchange.com/a/167354
https://unix.stackexchange.com/a/168407
https://unix.stackexchange.com/a/203170

> 
> my ideal would be something like the following; it requires giving -z options 
> to cut and tail (and head) and also creating a shortcut for the options that 
> make bash's `read' handle null-terminated text:
> 
> # ideal hypothetical code
> unset files
> typeset -a files
> dirs=(a b c)
> while read -z line
> do
>     files["${#files[@]}"]=$line
> done < <(for dir in "${dirs[@]}"; do find "$dir" -type f -printf '%T@\t%p\0' 
> | sort -zk1,1n | tail -zn1 | cut -z -d $'\t' -f 2; done)
> foo "${files[@]}"
[...]

(that should be cut -f 2-, note that bash also has a readarray
which in bash4.4 will support -d '' as well).

Yes, note with gawk and gsed supporting NUL delimited records, those
cut/head/tail could be done with them, but to me it makes little
sense to have some text utilities support NUL delimited records
and not all, that's more a consistency thing for me.

-- 
Stephane