Announcing the release of GNU findutils 4.6.0

[James Youngman]

I am pleased to announce the release of version 4.6.0 of GNU
findutils.

GNU findutils is a set of software tools for finding files that match
certain criteria and for performing various operations on them.
Findutils includes the programs "find", "xargs" and "locate".  More
information about findutils is available at
http://www.gnu.org/software/findutils/.

This is a "stable" release of findutils.  It can be downloaded from
ftp://ftp.gnu.org/pub/gnu/findutils/findutils-4.6.0.tar.gz.  The
ftp.gnu.org site is very busy, so you may find it more convenient to
download findutils from one of the mirror sites listed at
http://www.gnu.org/order/ftp.html.

Version 4.6.0 is based on a series of development releases (4.5.x).
All the changes and bugfixes since the previous stable release (4.4.2)
are summarised below.  Some of the bugs listed as fixed may not have
existed in any previous stable release; that is, some bugs listed as
fixed may have been introduced and then fixed during the lifetime of
the 4.5.x development releases.

Bugs in GNU findutils should be reported to the findutils bug tracker
at http://savannah.gnu.org/bugs/?group=findutils.  Reporting bugs via
the web interface will ensure that you are automatically informed when
the bug has been fixed.  General discussion of findutils takes place
on the bug-findutils mailing list.  To join the 'bug-findutils'
mailing list, send email to <address@hidden>.

To verify the GPG signature of the release, you will need the public
key of the findutils maintainer, James Youngman.  You can download
this from http://savannah.gnu.org/users/jay.

Please consider supporting the Free Software Foundation in its fund
raising appeal; see http://www.fsf.org/appeal/.


* Major changes in release 4.6.0, 2015-12-28

** Stable Release

This is the first stable release since findutils-4.4.2.  This release
includes all the bug fixes incorporated into the 4.4.x release series,
since those bug fixes were also applied to the 4.5.x release series.

** Summary

1. Some backward-incompatible changes have been made to find:
  - egrep regular expressions now work like GNU grep -E
  - Minor changes to the way nanoseconds fields are printed
  - find -perm +mode is now fully POSIX compliant (if you want the old
    behaviour use -perm /mode).
  - find -perm +numeric_mode is not supported any more. This syntax is
    unspecified by POSIX. The prior functionality continues to be
    available with -perm /numeric_mode. For more details see Savannah
    bug #38474.

2. Some backward-incompatible changes have been made to xargs:
  - if the child exits with status 126 or 127, xargs exits with status
    123.

3. There are also a large number of bugfixes, performance enhancements
   and documentation improvements.

4. The "oldfind" binary is no longer installed.

The changes are described in further detail below:

** Changes to find

** Removal of the recursive implementation of find

The oldfind binary used to be a recursive implementation, using one
file descriptor and at least 100 bytes of stack per nested
subdirectory.  This binary is still built but is no longer installed.
The configure option --without-fts is no longer accepted (it's
recognised but only --with-fts is allowed).

*** Regular Expressions

Using "-regextype egrep" now has the same effect as "-regextype
posix-egrep".  This is the result of a change to gnulib to bring it
into line with GNU grep (see
http://debbugs.gnu.org/cgi/bugreport.cgi?bug=20974#22).

The behaviour of the "awk", "posix-awk" and "gnu-awk" regular
expression types selected by the -regextype option also have slightly
changed, to bring them into line with the behaviour of the GNU C
library.  For "awk", character classes (such as [[:digit:]]) are now
supported.  For "gnu-awk" and "posix-awk", intervals are supported and
invalid interval specifcations are treated as literals (for example
'a{1' is treated as 'a\{1').

*** SELinux

SELinux is now supported.  Thanks to Miloslav Trmac and Kamil Dudka.

*** -perm

The obsolete GNU extension "find ... -perm +MODE", which was withdrawn
in release 4.2.21 in 2005 due to compatibility problems, has been
completely removed.  Use "find ... -perm /MODE" instead.

The GNU extension "find ... -perm /MODE" is no longer disabled when
the POSIXLY_CORRECT environment variable is set.

*** -printf

For find -printf, the format specifiers %{, %[ and %( are all now
reserved for future use.  Previously these would print {, [ and (
respectively, but in any case those characters can just be printed
literally like this: find -printf "{[(".  Code changes intended to
explain that these are reserved went into findutils-4.5.5, but this
code had, before now, had no effect.

When expanding "-printf '%F'", find reads /etc/mtab.  We now take the
last match found in this file, rather than the first, to better deal
with implementations which have duplicate entries (for example
/proc/mounts on systems running the Linux kernel).

*** -ok and -okdir

If the POSIXLY_CORRECT environment variable is set, the system's
definition of "yes" and "no" responses are used to interpret the
response to questions from -ok and -okdir.  The default is still to
use information from the findutils message translations.

*** Warnings

When the POSIXLY_CORRECT environment variable is set, a warning is no
longer issued when '/' is found in the argument to -name.  Use of
POSIXLY_CORRECT also turns off warnings about use of the deprecated
option -d  and the use of global options in surpising positions.

** Functional Changes to xargs

*** Default Command

If no utility is specified, xargs now calls "echo" (and searches on
$PATH to find it) rather than "/bin/echo".  This may give rise to
subtle behaviour differences for users having more than one echo
binary.  To avoid unexpected surprises, just explicitly specify the
utility you would like to run.  For example use "xargs /bin/echo <
foo" rather than "xargs < foo".

*** System Limits

If xargs find that exec fails because the argument size limit it
calculated is larger than the system's actual maximum, it now adapts
by passing fewer arguments (as opposed to failing).

Similarly, xargs now handles the case where the system's actual exec
limits are smaller than the value of ARG_MAX at compile time.  Very
few platforms normally have this property, but it is possible to
configure some Unix systems this way.

*** Exit Status

When the utility invoked from xargs exits with status 126 or 127,
xargs exits with status 123.  This means that exit status values 126
and 127 now unambigously mean that the utility could not be run or
could not be found, respectively.

*** Parallel Execution

You can now increase the parallelism of xargs in mid-run by sending
it SIGUSR1, and decrease the parallelism with SIGUSR2.

A new option is provided, --process-slot-var.  If you set this, xargs
will set the indicated environment variable in each child.  The values
are re-used, but no executing child process will have the same value
as another executing child process.

** Performance Changes to find

The find program will once again build argument lists longer than 1
with "-execdir ...+".  The upper limit of 1 argument for execdir was
introduced as a workaround in findutils-4.3.4.   The limit is now
removed, but find still does not issue the maximum possible number of
arguments, since an exec will occur each time find encounters a
subdirectory (if at least one argument is pending).

If you use the -fstype FOO predicate and specify a filsystem type FOO
which is not known (e.g. present in /etc/mtab) at the time find
starts, that predicate is now equivalent to -false.  This substitution
currently occurs at optimisation level 2 and above.

*** Optimization

The default optimisation level for find is now -O2 instead of -O0,
meaning that a number of additional optimisations are performed by
default.   Current optimisations at each level are:

0:      Perform -name, -path, -iname, -ipath before other checks.
1:      Expressions containing only cost-free tests are evaluated
        before expressions which contain more costly tests.
2:      Bring forward all tests that need to know the type of a file
        but don't need to stat it.
3:      All tests are ordered by their estimated cost.

Cost here is simply an estimate of how time consuming the I/O
operations needed to make a test are.

*** Calling stat fewer times

File type information is also passed back from fts to find, saving
calls to the stat system call for find command lines which don't need
the stat information.  This provides a performance improvement for
common cases like "find . -type d".

The ftsfind executable (which is built by default as "find") now calls
fts() in such a way that it avoids calling stat() on directory
entries, if it doesn't need the information.  This can produce a
significant speedup on filesystems which don't populate the d_type
element of struct dirent, for example reiserfs.  Anecdotal evidence
suggests this can speed updatedb up from about 30 minutes to 3-4
minutes.

The ftsfind executable also now avoids calling stat() functions to
discover the inode number of a file, if we already read this
information from the directory.  This does provide a speed-up, but
only for a restricted set of commands such as "find . -inum 4001".
This fix is listed below as bug #24342.

*** Large Directories

Find now uses less heap memory when processing directories containing
very many files.

Changes to gnulib's fts code should provide performance improvements
in find when processing very large directories (for example
directories containing significantly more than 10000 filenames).
Performance imporvements may only exist for some find command lines
(performance testing was done for the fts implementation itself but
we haven't done the analogous performance tests in find).

** Documentation Changes

The Texinfo manual now includes a small number of references to
further reading on security.

*** Documentation Changes for find

If you use -type or -xtype with a type letter corresponding to a file
type which is not supported by the system on which find was compiled,
find will now give a clearer error message (though the functionality
is unchanged).  Type letters are affected are D, l and p (for Solaris
Doors, symbolic links and named pipes respectively).

The documentation for find -execdir now describes correctly that the
command will be executed in the same directory as the file we were
considering at the time.  The documentation previously (and
incorrectly) stated that the original working directory of find would
be used.

Both the Texinfo manual and the find manual page now include a more
precise description of how your locale configuration affects the
interpretation of regular expressions and how your response to prompts
from the -ok action are interpreted.

*** Documentation Changes for xargs

The output of xargs --help has been slightly changed for greater
clarity.

The documentation for xargs now warns about parallel processes (xargs
-P) sharing stdout.

** Translations

There are now message translations for Norwegian Bokmaal, Czech and
Lithuanian.

** Bug Fixes

xargs now waits for a running process before prompting in interactive
mode (-p).

When the -a option of xargs is used, xargs no longer leaks a file
descriptor (fixing a bug reported by Kyle Sallee).

On some systems without support for a boolean type (for example some
versions of the AIX C compiler), find's regular expression
implementation fails to support case-insensitive regular expression
matching, causing -iregex to behave like -regex.  This is now fixed.

A large number of bugs from the Savanah bug tracker
(https://savannah.gnu.org/bugs/?group=findutils) have also been fixed:

#46715: testsuite error with perl 5.22, gnulib outdated

#45780: inode column is badly aligned when running 'find <dir> -ls'

#45585: unclear description of -newerXY in manual page.

#45505: give a more explicit error message when the argument to -regex
        is not a valid regular expression.

#45090: oldfind incorrectly omits test/..test (or any file whose name
        begins with ..).

#45065: find incorrectly prints a leading zero on the fractional part
        of ctime timestamps

#45064: Use of [[ ... ]] in /bin/sh script is incorrect

#45062: Enabling CACHE_IDS causes segfaults (this bug affects many
        historic releases, probably since release 3.0 in 1991).  You
        would not have been affected by this problem unless you used
        the option --enable-id-cache when invoking confgure.

#42903: checklists.py now supports Python 3.

#42793: "Failed to write output" with -ls (this bug affected only
        release 4.5.13).

#40805: The locatedb manual page uses now troff symbols where
        appropriate.

#40339: Fix leaked directory handle when listing mounted file
        systems.

#40146: gnulib revision doesn't support musl libc

#40094: The xargs --help output has a small number of cosmetic
        improvements.

#40088: potential buffer overflow in -execdir and -okdir

#39324: exits without error on OOM

#39197: Small fix to find's manual page to remove an unwanted
        backslash, which made the troff incompatible with Eric
        Raymond's doclifter software.

#39162: -printf reads beyond arguments terminated by \

#38583: errno-buffer read failed in xargs_do_exec

#38474: Unintended (?) behaviour change of -perm +mode predicate

#37926: The -inum predicate previously gave wrong results in oldfind
        (ftsfind, the default find binary, was unaffected).

#36652: Better document that -0/-d turns off the effect of -E.

#35753: Check the success/failure of material I/O operations where
        these are important to the use of the output (i.e. check the
        output for "find -ok" but not debugging output).

#34976: find -execdir leaks file descriptors for the working directory

#34079: Apply gnulib ftw memory fix

#33384: If rm/chmod etc. are not in /bin or /usr/bin, updatedb fails

#32887: Present xargs options alphabetically like in GNU cp(1) etc

#32043: find -name [ doesn't obey POSIX

#31359: test-strstr unit test fails on alpha.

#31005: The find manual page and Texinfo manual now more clearly state
        that -exec ... + always returns true.

#30777: find -exec echo TURNIP{} \+ is accepted but TURNIP is eaten

#30608: Automagic dependency on selinux.  The configure script now
        provides a --without-selinux option.

#30180: error message from incorrect -size option is off

#29949: find -execdir does not change working directory

#29828: test suite deadlock on FreeBSD.

#29698: Correct and clarify documentation of xargs -d option

#29593: Make import-gnulib.sh work under a POSIX shell.

#29511: fails to build on kfreebsd-*

#29460: -printf %Y fails in $CWD-dependent way

#29435: fd_is_cloexec does not work on Fedora buildhosts

#29089: SELinux --context and %Z options

#28872: Mistake in "#safer" example in "Problems with -exec and
        filenames" section of the Texinfo manual.

#28824: Corrected error message for "-ctime x".
        Likewise for -gid, -inum, -links, -mmin, -cmin, -amin,
        -uid, -used, -atime, -mtime, -ctime.

#27975: Infinite loop for -exec [..] {} +.

#27974: Use gnulib's xreadlinkat support

#27846: Assertion failure in xargs.c on AIX.

#27563: -L breaks -execdir

#27375: Open file descriptors leak into child processes.

#27328: segfault if the initial exec for "find -exec" fails.

#27221: symlink_loop check broken by FTS_CWDFD

#27213: avoid failed assertions for non-executable directories.

#27017: find -D opt / -fstype ext3 -print , -quit coredumps

#26868: compilation error in pred.c on Solaris x86_64

#26587: Fix a typo in -execdir documentation (it says -exec by mistake
        in the text).

#26537: find -prune now makes sure it has valid stat() information.

#26327: xargs man page is vague about the number of times command is executed.

#25764: remove duplicate entry for 'proc' in updatedb's $PRUNEFS.

#25359: find -H wrongly behaves like -L sometimes; this bug affects
       only filesystems which populate d_type and affects -type and
       -printf %y.  This does not affect the default behaviour of find
       or find -P.

#25154: Allow compilation with C compilers that don't allow
        declarations to follow statements.

#25144: Misleading error message when argument to find -user is an
        unknown user or is missing.

#24873: Duplicate fprint option corrupts output

#24342: -inum predicate shoud use dirent.d_ino instead of stat.st_ino
        (this is a performance bug).

#24283: -printf %TY causes NULL pointer dereference in find 4.5.2

#24169: find would segfault if the -newerXY test was not followed by
        any argument.

#23996: integer overflow on some platforms when parsing "-used 3".

#23920: warn about un-matchable -path arguments ending in /.

#23663: crash in some locales for -printf %AX (this problem seems to
        have affected only the CVS code, and not any public releases).

#23070: Corrected manpage description of find -perm /000 (the change
  was already made but the manpage indicated the change would
  happen "soon").

#22708: Exit status 126 and 127 from the utility invoked from xargs
now makes xargs return 123, meaning that exit status values
126 and 127 now unambigously mean that the utility could not
be run or could not be found, respectively.

#22662: nanoseconds wrongly appended after "PM" for find -printf %AX
        in locale en_US.UTF-8.

#20873: Doc bugfix: Indicate that * matches / and leading dot in
        filenames for "find -path".

#19593: -execdir .... {} + has suboptimal performance (see
        "Performance Changes to find".).

#19120: Patch to fix single quotes in man page find(1)

#18227: find -ls does not display device major/minor numbers.

#15472: Error messages that print ino_t values are no longer truncated
        on platforms with 64-bit ino_t.

#14386: updatedb relies on mktemp, which is not portable.


--
James Youngman <address@hidden>
GNU findutils maintainer