Re: RFC: Building a Shared Library (take 2)

[Alexandre Duret-Lutz]

Here is a second version of this section, so that we have
something cleaner in the archives.  Beside including comments
from Guido, Harlan, and Tim, I've split the whole section into
shorter nodes (it was all on one node), and added a section
about `ltmain.sh not found'.

(Tim I'll post the Texinfo code you asked later tonight on
automake-patches along with the test cases.)


Building a Shared Library
=========================

Building shared libraries portably is a relatively complex matter.  For
this reason, GNU Libtool (*note Introduction: (libtool)Top.) was
created to help build shared libraries in a platform-independent way.

* Menu:
  
* Libtool Concept::             Introducing Libtool
* Libtool Libraries::           Declaring Libtool Libraries
* Conditional Libtool Libraries::  Building Libtool Libraries Conditionally
* Conditional Libtool Sources::  Choosing Library Sources Conditionally
* Libtool Convenience Libraries::  Building Convenience Libtool Libraries
* Libtool Modules::             Building Libtool Modules
* Libtool Flags::               Using _LIBADD and _LDFLAGS
* LTLIBOBJ::                    Using $(LTLIBOBJ)
* Libtool Issues::              Common Issues Related to Libtool's Use

The Libtool Concept
-------------------

Libtool abstracts shared and static libraries into a unified concept
henceforth called "libtool libraries".  Libtool libraries are files
using the `.la' suffix, and can designate a static library, a shared
library, or maybe both.  Their exact nature cannot be determined until
`./configure' is run: not all platforms support all kinds of libraries,
and users can explicitly select which libraries should be built.
(However the package's maintainers can tune the default, *Note The
`AC_PROG_LIBTOOL' macro: (libtool)AC_PROG_LIBTOOL.)

   Because object files for shared and static libraries must be compiled
differently, libtool is also used during compilation.  Object files
built by libtool are called "libtool objects": these are files using
the `.lo' suffix.  Libtool libraries are built from these libtool
objects.

   People considering writing a plug-in system, with dynamically loaded
modules, should look into `libltdl': libtool's dlopening library (*note
Using libltdl: (libtool)Using libltdl.).  This offers a portable
dlopening facility to load libtool libraries dynamically, and can also
achieve static linking where unavoidable.

   Before we discuss how to use libtool with Automake in details, it
should be noted that the libtool manual also has a section about how to
use Automake with libtool (*note Using Automake with Libtool:
(libtool)Using Automake.).

Building Libtool Libraries
--------------------------

Automake uses libtool to build libraries declared with the
`LTLIBRARIES' primary.  Each `_LTLIBRARIES' variable is a list of
libtool libraries to build.  For instance, to create a libtool library
named `libgettext.la', and install it in `libdir', write:

     lib_LTLIBRARIES = libgettext.la
     libgettext_la_SOURCES = gettext.c gettext.h ...

   Automake predefines the variable `pkglibdir', so you can use
`pkglib_LTLIBRARIES' to install libraries in `$(libdir)/@PACKAGE@/'.

Building Libtool Libraries Conditionally
----------------------------------------

Like conditional programs (*note Conditional Programs::), there are two
main ways to build conditional libraries: using Automake conditionals
or using Autoconf `AC_SUBST'itutions.

   The important implementation detail you have to bwe aware of is that
the place where a library will be installed matters to libtool: it
needs to be indicated _at link-time_ using the `-rpath' option.

   For libraries whose destination directory is known when Automake
runs, Automake will automatically supply the appropriate `-rpath'
option to libtool. This is the case for libraries listed explicitly in
some `DIR_LTLIBRARIES' variable.

   However, for libraries determined at configure time (and thus
mentioned in `EXTRA_LTLIBRARIES'), Automake does not know the final
installation directory.  For such libraries you must add the `-rpath'
option to the appropriate `_LDFLAGS' variable by hand.

   The examples below illustrate the differences between these two
methods.

   Here is an example where `$(WANTEDLIBS)' is an `AC_SUBST'ed variable
set at `./configure'-time to either `libfoo.la', `libbar.la', both, or
none.  Although `$(WANTEDLIBS)' appears in the `lib_LTLIBRARIES',
Automake cannot guess it relates to `libfoo.la' or `libbar.la' by the
time it creates the link rule for these two libraries.  Therefore the
`-rpath' argument must be explicitly supplied.

     EXTRA_LTLIBRARIES = libfoo.la libbar.la
     lib_LTLIBRARIES = $(WANTEDLIBS)
     libfoo_la_SOURCES = foo.c ...
     libfoo_LDFLAGS = -rpath '$(libdir)'
     libbar_la_SOURCES = bar.c ...
     libbar_LDFLAGS = -rpath '$(libdir)'

   Here is how the same `Makefile.am' would look using Automake
conditionals named `WANT_LIBFOO' and `WANT_LIBBAR'.  Now Automake is
able to compute the `-rpath' setting itself, because it's clear that
both libraries will end up in `$(libdir)' if they are installed.

     lib_LTLIBRARIES =
     if WANT_LIBFOO
     lib_LTLIBRARIES += libfoo.la
     endif
     if WANT_LIBBAR
     lib_LTLIBRARIES += libbar.la
     endif
     libfoo_la_SOURCES = foo.c ...
     libbar_la_SOURCES = bar.c ...

Libtool Libraries with Conditional Sources
------------------------------------------

Conditional compilation of sources in a library can be achieved in the
same way as conditional compilation of sources in a program (*note
Conditional Sources::).  The only difference is that `_LIBADD' should
be used instead of `_LDADD' and that it should mention libtool objects
(`.lo' files).

   So, to mimic the `hello' example from *Note Conditional Sources::,
we could build a `libhello.la' library using either `hello-linux.c' or
`hello-generic.c' with the following `Makefile.am'.

     lib_LTLIBRARIES = libhello.la
     libhello_la_SOURCES = hello-common.c
     EXTRA_libhello_la_SOURCES = hello-linux.c hello-generic.c
     libhello_la_LIBADD = $(HELLO_SYSTEM)
     libhello_la_DEPENDENCIES = $(HELLO_SYSTEM)

And make sure `$(HELLO_SYSTEM)' is set to either `hello-linux.lo' or
`hello-generic.lo' in `./configure'.

   Or we could simply use an Automake conditional as follows.

     lib_LTLIBRARIES = libhello.la
     libhello_la_SOURCES = hello-common.c
     if LINUX
     libhello_la_SOURCES += hello-linux.c
     else
     libhello_la_SOURCES += hello-generic.c
     endif

Libtool Convenience Libraries
-----------------------------

Sometimes you want to build libtool libraries which should not be
installed.  These are called "libtool convenience libraries" and are
typically used to encapsulate many sublibraries, later gathered into
one big installed library.

   Libtool convenience libraries are declared by `noinst_LTLIBRARIES',
`check_LTLIBRARIES', or even `EXTRA_LTLIBRARIES'.  Unlike installed
libtool libraries they do not need an `-rpath' flag at link time
(actually this is the only difference).

   Convenience libraries listed in `noinst_LTLIBRARIES' are always
built.  Those listed in `check_LTLIBRARIES' are built only upon `make
check'.  Finally, libraries listed in `EXTRA_LTLIBRARIES' are never
built explicitly: Automake outputs rules to build them, but if the
library does not appear as a Makefile dependency anywhere it won't be
built (this is why `EXTRA_LTLIBRARIES' is used for conditional
compilation).

   Here is a sample setup merging libtool convenience libraries from
subdirectories into one main `libtop.la' library.

     # -- Top-level Makefile.am --
     SUBDIRS = sub1 sub2 ...
     lib_LTLIBRARIES = libtop.la
     libtop_la_SOURCES =
     libtop_la_LIBADD = \
       sub1/libsub1.la \
       sub2/libsub2.la \
       ...
     
     # -- sub1/Makefile.am --
     noinst_LTLIBRARIES = libsub1.la
     libsub1_la_SOURCES = ...
     
     # -- sub2/Makefile.am --
     # showing nested convenience libraries
     SUBDIRS = sub2.1 sub2.2 ...
     noinst_LTLIBRARIES = libsub2.la
     libsub2_la_SOURCES =
     libsub2_la_LIBADD = \
       sub2.1/libsub1.2.la \
       sub2.2/libsub2.2.la \
       ...

Libtool Modules
---------------

These are libtool libraries meant to be dlopened.  They are indicated
to libtool by passing `-module' at link-time.

     pkglib_LTLIBRARIES = mymodule.la
     mymodule_la_SOURCES = doit.c
     mymodule_LDFLAGS = -module

   Ordinarily, Automake requires that a Library's name starts with
`lib'.  However, when building a dynamically loadable module you might
wish to use a "nonstandard" name.

_LIBADD and _LDFLAGS
--------------------

As shown in previous sections, the `LIBRARY_LIBADD' variable should be
used to list extra libtool objects (`.lo' files) or libtool libraries
(`.la') to add to LIBRARY.

   The `LIBRARY_LDFLAGS' variable is the place to list additional
libtool flags, such as `-version-info', `-static', and a lot more.  See
*Note Using libltdl: (libtool)Link mode.

`LTLIBOBJS'
-----------

Where an ordinary library might include `$(LIBOBJS)', a libtool library
must use `$(LTLIBOBJS)'.  This is required because the object files
that libtool operates on do not necessarily end in `.o'.

   Nowadays, the computation of `LTLIBOBJS' for `LIBOBJS' is performed
automatically by Autoconf (*note `AC_LIBOBJ' vs. `LIBOBJS':
(autoconf)AC_LIBOBJ vs LIBOBJS.).

Issues Related to Libtool's Use
-------------------------------

`required file `./ltmain.sh' not found'
.......................................

Libtool comes with a tool called `libtoolize' that will install
libtool's supporting files into a package.  Running this command will
install `ltmain.sh'.  You should execute it before `aclocal' and
`automake'.

   People upgrading old packages to newer autotools are likely to face
this issue because older Automake versions used to call `libtoolize'.
Therefore old build scripts do not call `libtoolize'.

   Since Automake 1.6, it has been decided that running `libtoolize'
was none of Automake's business.  Instead, that functionality has been
moved into the `autoreconf' command (*note Using `autoreconf':
(autoconf)autoreconf Invocation.).  If you do not want to remember what
to run and when, just learn the `autoreconf' command.  Hopefully,
replacing existing `bootstrap.sh' or `autogen.sh' scripts by a call to
`autoreconf' should also free you from any similar incompatible change
in the future.

Objects `created with both libtool and without'
...............................................

Sometimes, the same source file is used both to build a libtool library
and to build another non-libtool target (be it a program or another
library).

   Let's consider the following `Makefile.am'.

     bin_PROGRAMS = prog
     prog_SOURCES = prog.c foo.c ...
     
     lib_LTLIBRARIES = libfoo.la
     libfoo_la_SOURCES = foo.c ...

(In this trivial case the issue could be avoided by linking `libfoo.la'
with `prog' instead of listing `foo.c' in `prog_SOURCES'.  But let's
assume we really want to keep `prog' and `libfoo.la' separate.)

   Technically, it means that we should build `foo.$(OBJEXT)' for
`prog', and `foo.lo' for `libfoo.la'.  The problem is that in the
course of creating `foo.lo', libtool may erase (or replace)
`foo.$(OBJEXT)' - and this cannot be avoided.

   Therefore, when Automake detects this situation it will complain
with a message such as
     object `foo.$(OBJEXT)' created both with libtool and without

   A workaround for this issue is to ensure that these two objects get
different basenames.  As explained in *Note renamed objects::, this
happens automatically when per-targets flags are used.

     bin_PROGRAMS = prog
     prog_SOURCES = prog.c foo.c ...
     prog_CFLAGS = $(AM_CFLAGS)
     
     lib_LTLIBRARIES = libfoo.la
     libfoo_la_SOURCES = foo.c ...

Adding `prog_CFLAGS = $(AM_CFLAGS)' is almost a no-op, because when the
`prog_CFLAGS' is defined, it is used instead of `AM_CFLAGS'.  However
as a side effect it will cause `prog.c' and `foo.c' to be compiled as
`prog-prog.$(OBJEXT)' and `prog-foo.$(OBJEXT)' which solves the issue.

-- 
Alexandre Duret-Lutz