[Top][All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[Automake-NG] [PATCH 3/6] [ng] am.xargs-map: be usable with huge lists
From: |
Stefano Lattarini |
Subject: |
[Automake-NG] [PATCH 3/6] [ng] am.xargs-map: be usable with huge lists |
Date: |
Sat, 7 Jul 2012 22:40:44 +0200 |
The previous implementation could eat all memory if the given list was
really huge -- but this function will be needed to operate on potentially
huge lists! (i.e., list of tests or files which can exceed the command-line
length limit of the system).
As a drawback, the function cannot recursively invoke itself anymore (not
even indirectly), but this is an acceptable limitation for its intended
uses.
* lib/am/header-vars.am (am.xargs-map): Reimplemented.
(am.max-cmdline-args): Redefine, as its semantics have changed.
(am.max-cmdline-args+1): Remove, no more needed.
Signed-off-by: Stefano Lattarini <address@hidden>
---
lib/am/header-vars.am | 31 +++++++++++++++++++++++--------
1 file changed, 23 insertions(+), 8 deletions(-)
diff --git a/lib/am/header-vars.am b/lib/am/header-vars.am
index 812f2f6..d7afe6f 100644
--- a/lib/am/header-vars.am
+++ b/lib/am/header-vars.am
@@ -259,24 +259,39 @@ am__strip_suffixes = $(strip \
# Helper variables and function to help in recipes that could exceed
# the command line length limit.
-## FIXME: this is basically arbitrary. In the long term, defining this
-## FIXME: after a configure-time test on the command-line length limits,
-## FIXME: or at least on a system-by-system basis, might be better.
-am.max-cmdline-args := 40
-am.max-cmdline-args+1 := 41
+## FIXME: Forty aguments; this is basically arbitrary. In the long term,
+## FIXME: defining this after a configure-time test on the command-line
+## FIXME: length limits, or at least on a system-by-system basis, might
+## FIXME: be better. But don't make it too big, or our implementation
+## FIXME: will likely suffer in performance and memory consumption.
+# And in the Information Age, we somehow managed to revert an abacus-like
+# counting. Yay for us :-)
+am.max-cmdline-args := xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
# $(call am.xargs-map,FUNCTION,LIST)
# ----------------------------------
# Map the function $1 on the arguments $2, ensuring that each
# call of $1 has at most 40 arguments.
+# This implementation is hacky, but the more elegant or "naive" ones
+# (based on recursion) proved to be ludicrously memory-hungry with
+# huge lists.
+# A drawback of this implementation is that am.xargs-map cannot be
+# recursively invoked, but that shouldn't matter for our use cases.
# The extra $(strip) calls are only to allow clearer formatting.
define am.xargs-map
$(if $2,$(strip \
- )$(call $1,$(wordlist 1,$(am.max-cmdline-args),$2))$(strip \
- )$(call $0,$1,$(wordlist $(am.max-cmdline-args+1),$(words $2),$2)))
+ )$(eval $0.partial-args :=)$(strip \
+ )$(eval $0.counter :=)$(strip \
+ )$(foreach i,$2,$(strip \
+ )$(eval $0.counter := $$($0.counter)x)$(strip \
+ )$(eval $0.partial-args += $$i)$(strip \
+ )$(if $(filter $(am.max-cmdline-args),$($0.counter)),$(strip \
+ )$(call $1,$(strip $($0.partial-args)))$(strip \
+ )$(eval $0.partial-args :=)$(strip \
+ )$(eval $0.counter :=)))$(strip \
+ )$(if $($0.counter),$(call $1,$(strip $($0.partial-args)))))
endef
-
## Some derived variables that have been found to be useful.
pkgdatadir = $(datadir)/@PACKAGE@
pkgincludedir = $(includedir)/@PACKAGE@
--
1.7.9.5
[Automake-NG] [PATCH 6/6] [ng] clean: do not exceed command line length limits, even with many files, Stefano Lattarini, 2012/07/07
Re: [Automake-NG] [PATCH 0/6] clean: avoid issues with excessive command line length, Akim Demaille, 2012/07/09