bug#19614: Split packaging invocation to catch errors

[Nick Bowler]

On 2023-07-17, Karl Berry <karl@freefriends.org> wrote:
> Hi Dimitrios, Bogdan - back on this bug from 2015 (sorry):
>     https://debbugs.gnu.org/cgi/bugreport.cgi?bug=19614
>
> Bogdan sent a patch that splits the tar and compress into separate
> invocations.  It seems basically good to me, but the dist-formats test
> fails because it builds multiple archive formats (.tar.gz, .tar.bz2,
> etc.) in parallel, and so removing $(distdir).tar (and .err) files are
> subject to a race condition.

>  dist-gzip: distdir
> -     tardir=$(distdir) && $(am__tar) | eval GZIP= gzip $(GZIP_ENV) -c
>>$(distdir).tar.gz
> +     tardir=$(distdir) && $(am__tar) > $(distdir).tar 2>$(distdir).tarerr
>
> So my question is, will it suffice in this limited case to just put $$
> into the filenames? It seems like it should be ok to me, but I'm not
> sure I have enough imagination to know why that would fail. I can't see
> figuring out how to run mktemp here.

With the tar file generation as a separate command, it should be
straightforward to avoid this problem by just moving the tar generation
and error checking commands into a separate rule.  Then changing all the
various dist-xyz commands to depend on that instead of distdir.  Example:

  $(distdir).tar: distdir
        commands to tar it

  dist-gzip: $(distdir).tar
        commands to gzip it

and so on.  Then there should be no race with parallel "make dist" as
the tar file will only be generated once.

Additional context/suggestion (not directly relevant to this bug report):

Note that "make dist" (parallel or othewrise) with multiple compressors
enabled only works because "make dist" itself does a recursive make
invocation and overrides am__post_remove_distdir.  Otherwise the cleanup
commands would conflict too.  It's quite hacky, and as a result it
currently doesn't work to explicitly call multiple dist targets in a
single make invocation, e.g.,

  make dist-zip dist-gzip # fails

Using a separate rule to generate the tar file should let us fix this
problem pretty easily too, should just be a matter of:

  - do not add "rm -f $(distdir).tar" to am__post_remove_distdir
  - arrange for make dist and make clean to delete $(distdir).tar
  - add the rule .INTERMEDIATE: $(distdir).tar

The last item instructs GNU make to delete the temporary tar file in
normal cases once all the rules that depend on it have been run.

Explicit deletion in "make dist" and "make clean" is then mostly for the
benefit of other implementations.  On other makes, the tar file will be
left behind in some cases (e.g., if the user runs make dist-zip).  This
makes a change from current behaviour but probably not a real problem.

Cheers,
  Nick