FYI: handling tools that produce many outputs (FAQ)

[Alexandre Duret-Lutz]

I'm installing this on HEAD and branch-1-8.

2004-02-01  Alexandre Duret-Lutz  <address@hidden>

        * doc/automake.texi (multiple outputs): New node.

Index: NEWS
===================================================================
RCS file: /cvs/automake/automake/NEWS,v
retrieving revision 1.256.2.23
diff -u -r1.256.2.23 NEWS
--- NEWS        31 Jan 2004 14:36:19 -0000      1.256.2.23
+++ NEWS        1 Feb 2004 09:27:22 -0000
@@ -21,6 +21,7 @@
 
   - Third-Party Makefiles: how to interface third party Makefiles.
   - Upgrading: upgrading packages to newer Automake versions.
+  - Multiple Outputs: handling tools that produce many outputs.
 
 Bug fixed in 1.8.2:
 
Index: doc/automake.texi
===================================================================
RCS file: /cvs/automake/automake/doc/automake.texi,v
retrieving revision 1.18.2.6
diff -u -r1.18.2.6 automake.texi
--- doc/automake.texi   28 Jan 2004 17:09:21 -0000      1.18.2.6
+++ doc/automake.texi   1 Feb 2004 09:27:29 -0000
@@ -244,6 +244,7 @@
 * wildcards::                   Why doesn't Automake support wildcards?
 * distcleancheck::              Files left in build directory after distclean
 * renamed objects::             Why are object files sometimes renamed?
+* Multiple Outputs::            Writing rules for tools with many output files
 
 Copying This Manual
 
@@ -6727,6 +6728,7 @@
 * wildcards::                   Why doesn't Automake support wildcards?
 * distcleancheck::              Files left in build directory after distclean
 * renamed objects::             Why are object files sometimes renamed?
+* Multiple Outputs::            Writing rules for tools with many output files
 @end menu
 
 @node CVS
@@ -7235,6 +7237,127 @@
 
 Note that the renaming of objects is also affected by the
 @code{_SHORTNAME} variable (@pxref{Program and Library Variables}).
+
+
address@hidden Multiple Outputs
address@hidden Handling Tools that Produce Many Outputs
address@hidden multiple outputs, rules with
address@hidden many outputs, rules with
address@hidden rules with multiple outputs
+
+This section describes a @command{make} idiom that can be used when a
+tool produces multiple outputs.  It is not specific to Automake and
+can be used in ordinary @file{Makefile}s.
+
+Suppose we have a program called @command{foo} that will read one file
+called @file{data.foo} and produce two files called @file{data.c} and
address@hidden  We want to write a @file{Makefile} rule that captures
+this one-to-two dependency.
+
+The naive rule is incorrect:
+
address@hidden
+# This is incorrect.
+data.c data.h: data.foo
+        foo data.foo
address@hidden example
+
address@hidden
+What the above rule really says is that @file{data.c} and
address@hidden each depend on @file{data.foo}, and can each be built by
+running @code{foo data.foo}.  In other words it is equivalent to:
+
address@hidden
+# We do not want this.
+data.c: data.foo
+        foo data.foo
+data.h: data.foo
+        foo data.foo
address@hidden example
+
address@hidden
+which means that @command{foo} can be run twice.  It will not
address@hidden run twice, because many @command{make}
+implementations will check for the second file after the first one has
+been built and will therefore detect that it already exists.  However
+it can run twice, and we should avoid that.  An easy way to trigger
+the problem is to run a parallel make; if @file{data.c} and
address@hidden are built in parallel, two @code{foo data.foo}
+commands will run concurrently.
+
+Another idea is to write the following:
+
address@hidden
+# There is still a problem with this one.
+data.c: data.foo
+        foo data.foo
+data.h: data.c
address@hidden example
+
address@hidden
+The idea is that @code{foo data.foo} is run only when @file{data.c}
+need to be updated, but we further state that @file{data.h} depends
+upon @file{data.c}.  That way, if @file{data.h} is required and
address@hidden is out of data, the dependency on @file{data.c} will
+trigger the build.
+
+This is almost perfect, but suppose we have built @file{data.h} and
address@hidden, and then we erase @file{data.h}.  Then, running
address@hidden data.h} will not rebuild @file{data.h}.  The above rules
+just state that @file{data.c} must be up-to-date with respect to
address@hidden, and this is the case.
+
+What we need is a rule that forces a rebuild when data.h is missing.
+
address@hidden
+data.c: data.foo
+        foo data.foo
+data.h: data.c
+        @@if test -f $@@; then :; else \
+          rm -f data.c; \
+          $(MAKE) $(AM_MAKEFLAGS) data.c; \
+        fi
address@hidden example
+
+The above scales easily to more outputs and more inputs.  For instance
+if @command{foo} should read @file{data.bar} and will also produce
address@hidden and @file{data.x}, we would write:
+
address@hidden
+data.c: data.foo data.bar
+        foo data.foo data.bar
+data.h data.w data.x: data.c
+        @@if test -f $@@; then :; else \
+          rm -f data.c; \
+          $(MAKE) $(AM_MAKEFLAGS) data.c; \
+        fi
address@hidden example
+
+One of the output files (here @file{data.c}) is used as a witness of
+the run of @command{foo}.  The other files depend upon that witness.
+Ideally the witness should have the oldest timestamp among the output
+files, so that the second rule (@code{data.h data.w data.x: data.c})
+is not triggered needlessly.  For this reason, it is often better to
+use a different file (not one of the output files) as witness.
+
address@hidden
+data.stamp: data.foo data.bar
+        @@rm -f data.tmp
+        @@touch data.tmp
+        foo data.foo data.bar
+        @@mv -f data.tmp $@@
+data.c data.h data.w data.x: data.stamp
+        @@if test -f $@@; then :; else \
+          rm -f data.stamp; \
+          $(MAKE) $(AM_MAKEFLAGS) data.stamp; \
+        fi
address@hidden example
+
address@hidden is created before @command{foo} is run, so that is has
+a timestamp older than output files output by @command{foo}.  It is
+then renamed to @file{data.stamp} after @command{foo} has run, because
+we do not want to update @file{data.stamp} if @command{foo} fails.
+
 
 @c ========================================================== Appendices
 
-- 
Alexandre Duret-Lutz