Re: * limitation?

[Bob Proulx]

> Basically, I've occasionally have problems with the limit imposed by the
> "*" operator when handling files. E.g.

You are running up against the kernel ARG_MAX limit.  See this pointer
for details.

  http://www.gnu.org/software/fileutils/doc/faq/#Argument%20list%20too%20long

> address@hidden 2002-replaced]$ rm -f *
> bash: /bin/rm: Argument list too long

Yes.  This is one of the tradeoffs that was made in the kernel.  Not
all memory is completely dynamic.  Try this, at your own risk of
course.  I put the echo in there because the command scares me and
should scare you too.  :-)

  ls | xargs echo rm -f

> address@hidden 2002-replaced]$ ls | wc -l
>  231937 (yeah yeah I know!)

Wow!

> address@hidden 2002-replaced]$ ls * | wc -l
> bash: /bin/ls: Argument list too long

  getconf ARG_MAX

The kernel gives you a buffer of that size.  The shell tries to put
the contents of the '*' glob expansion into that buffer.  It can't.
The argument list is too long.  It fails.  The ls command is not run
in that case.

> I find it interesting that the `ls' works for an obscene number of
> files, yet `ls *' doesn't. bash fails nicely as above, but /bin/sh
> either returns "Segmentation fault" or kills my connection!

On which systems?  "Segmentation fault" is a bug to be reported.  On
GNU systems /bin/sh is bash and so I suspect you are not on a GNU OS
but on a different OS with bash ported to it?  In either case if
/bin/sh is dieing with a SIGSEGV then I would report it as a bug.

As an aside, if you suspect that a shell problem will kill the shell,
then stack another one first.

  $ sh                   # stack a shell
  $ ls *                 # possible killer command
  $ Segmentation fault   # shell died
  $                      # Back to the first shell and not logged out.

That is a good way to avoid getting logged out while testing.

> Any ideas (other than "don't get involved with legacy systems that store
> lots of files in a single dir")?

Well, that would be good for other reasons.  Most systems that have
cases where things like that are needed use the first letter or
frequently the first two letters as the directory that the file is
stored into.  Things like that are terminfo directories and mail spool
directories and web cache directories and things like that.

Bob