[Top][All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
xargs: exec's with too many arguments
From: |
Jim Meyering |
Subject: |
xargs: exec's with too many arguments |
Date: |
Fri, 18 Nov 2005 14:54:36 +0100 |
address@hidden (James Youngman) wrote:
...
> Many systems now include the 'buffer' program (I have it here on an
> Ubuntu box). If you have it, try piping the data through 'buffer -z
> 4096' (or whatever PIPE_BUF is on your machine if it's not 4096).
Hi James,
Funny seeing a message from you, then minutes later seeing the symptom
of a bug in xargs. I was running xargs to grep through the files in an
mh-style folder and got an `Argument list too long' error.
Here's a stand-alone example to demonstrate the problem:
$ seq 100000|/usr/bin/xargs grep foo
/usr/bin/xargs: grep: Argument list too long
[Exit 126]
$ /usr/bin/xargs --version
GNU xargs version 4.2.25
This is on a 64-bit system running linux-2.4.16.
Here's a tighter example:
$ yes|head -n 16382|tr '\n' ' ' |xargs echo > /dev/null
$ yes|head -n 16383|tr '\n' ' ' |xargs echo > /dev/null
xargs: echo: Argument list too long
[Exit 126]
With the patch included below, it works:
$ yes|head -n 16383|tr '\n' ' ' |./xargs echo > /dev/null && echo ok
ok
You can demonstrate the bug on 32-bit systems too,
but the limits are higher, so for normal usage (with non-trivial
arguments), it's a lot less likely to trigger:
g$ yes|head -n 32766|tr '\n' ' '|xargs echo > /dev/null && echo ok
ok
g$ yes|head -n 32767|tr '\n' ' '|xargs echo
xargs: echo: Argument list too long
[Exit 126]
This is because Linux's execve implementation requires that the sum of
the sizes of all argument string pointers not exceed 128K (the actual
limit is ARG_MAX - sizeof (void*)). The difference between 32- and
64-bit systems is because sizeof (void*) differs (4 vs. 8).
2005-11-18 Jim Meyering <address@hidden>
* lib/buildcmd.c (bc_push_arg): When exec'ing, don't exceed
Linux's limit on the maximum number of command line arguments.
Index: lib/buildcmd.c
===================================================================
RCS file: /cvsroot/findutils/findutils/lib/buildcmd.c,v
retrieving revision 1.9
diff -u -p -r1.9 buildcmd.c
--- lib/buildcmd.c 2 Jul 2005 09:54:34 -0000 1.9
+++ lib/buildcmd.c 18 Nov 2005 13:48:17 -0000
@@ -250,9 +250,10 @@ bc_push_arg (const struct buildcmd_contr
* conditional on arg!=NULL, since do_exec()
* actually calls bc_push_arg(ctl, state, NULL, 0, false).
*/
- if (!initial_args
- && ctl->args_per_exec
- && (state->cmd_argc - ctl->initial_argc) == ctl->args_per_exec)
+ if ((!initial_args
+ && ctl->args_per_exec
+ && (state->cmd_argc - ctl->initial_argc) == ctl->args_per_exec)
+ || state->cmd_argc == ARG_MAX / sizeof (void *) - 1)
do_exec (ctl, state);
}
- xargs: exec's with too many arguments,
Jim Meyering <=