Non-recursive makefiles

[Olly Betts]

I've been looking at the feasibility of converting a project (Xapian)
using autoconf+automake+libtool to using non-recursive makefiles.  

Currently each subdirectory produces a libtool convenience library
and these are linked into the main installable library.  There are
a few convenience libraries which are linked into binaries directly
(e.g. there's a libgetopt.la).  The main drawback of this approach
is that the convenience libraries add a lot of space bloat to a built
tree - the object files end up essentially duplicated in the convenience
libraries, and where there are sub-subdirectories, they get duplicated
twice!  I suspect building all these convenience libraries adds some
time to the build too.

Looking at the list archives, the basic idea usually suggested is
to include Makefile.am snippets from each subdirectory into the
top-level Makefile.am.  These snippets either define variables
which the top-level Makefile.am combines or they just directly
use += to add to variables set (possibly to empty) before including
the snippets.

I've tried converting a few subdirectories and the results are
promising.  Configure is a few percent faster thanks to having fewer
Makefile.in-s to substitute, builds are a little faster thanks to fewer
invocations of make and not having to build so many convenience
libraries, and dependency tracking is better (with a simple makefile in
each converted directory "make" in a subdirectory now rebuilds
correctly).

I'm fairly convinced it'll be an improvement overall, but there are a
few issues I've noticed so far:

* Generally, it would be useful for the manual to go into a bit more
  detail about how to approach all this.

* I'm not sure what to call the Makefile.am snippets in the
  subdirectories.  I looked at GraphicsMagick which was suggested to be
  a good example in a previous thread and that calls them "Makefile.am".
  That helps editors know how to syntax highlight, but it feels odd to
  me and I wonder if it might confuse people to see "Makefile.am"
  without "Makefile.in", especially if there's an unrelated static
  "Makefile".

  I also worry that it could confuse the autotools (or may in some
  future version.)  Perhaps the automake manual section on non-recursive
  use (and/or the description of "include") should either bless naming
  the included snippets "Makefile.am", or recommend a way to name
  included files?

* The snippets turn what used to be:

  libfoo_la_SOURCES = foo.c bar.c baz.c

  into:

  libmain_la_SOURCES += foo/foo.c foo/bar.c foo/baz.c

  which I think for long lists becomes significantly less readable and
  harder to maintain.  It'd be really handy to have a way to
  automatically prepend the directory that the current snippet was
  included with.  So after "include foo/Makefile.am", I can write:

  libmain_la_SOURCES +/= foo.c bar.c baz.c

  and that would be equivalent to the version above.  "+/=" looks odd
  but doesn't collide with anything currently valid, and would allow
  ":/=" and "/=" to do the same magic to ":=" and "=".  I'm not at all
  wedded to this syntax, it's just the best that's occurred to me so far.

  If this seems a good idea I'm happy to prepare a patch (I've already
  had a poke around and it shouldn't be too hard to do.)

* I have a subdirectory which contains generated code which I currently
  disable warnings for.  Is there an easy way to specify different flags
  for a set of objects without putting them in a convenience library or
  keeping them in their own recursively made directory?

* It would be nice to have a standard feature to generate stub Makefiles
  in the non-longer-recursed directories which would at least support
  "make all", "make check", and "make clean".  Ideally these want to be in
  the build-tree so they're available when srcdir != builddir.  Perhaps
  it'll be tricky to generate the list of files to clean though?  Or
  perhaps "make clean" in a subdirectory should clean the whole tree?

  Currently I've just added this Makefile to each non-recursed source
  directory and experimented with building in the source tree:

    SHELL = /bin/sh
 
    all check:
            cd .. && $(MAKE) $(MAKEFLAGS) $@
 
    clean:
            rm -f *.o *.obj *.lo

  I could just generate these using AC_OUTPUT (so they end up in the
  build tree) and maintain the lists of files to clean by hand, but
  then there's a risk of failing to clean an object which should be
  cleaned which could lead to frustration.

Cheers,
    Olly