Re: how to turn off shared library notice in output of make install?

[Kent Boortz]

Ralf Wildenhues <address@hidden> writes:
<snip>
> | ----------------------------------------------------------------------
> | Libraries have been installed in:
> |    /shecky/n3_new/install/lib
> | 
> | If you ever happen to want to link against installed libraries
> | in a given directory, LIBDIR, you can use libtool, and specify
> | the full pathname of the library, or use the `-LLIBDIR'
> | flag during linking.
> | ----------------------------------------------------------------------
>
> Or would people prefer that nothing be output in this case?

I would prefer very short warnings with a link to a section in an
appendix in the manual that in depth explain what the warning really
means, in a language even I can understand ;-)

I think the GNU autotools are such great stuff, but as it is attacking
a hard problem it needs lots of documentation. The amount of
flexibility given enable you to shoot yourself in the foot in so many
ways. And there is a difference between "what you can do", and "what
you should do", knowledge hard to gain even if googling a lot.

I have been toying with an idea to create a wiki or something that is
a GNU autotools "cookbook". Not a tutorial, but more like the Perl
cookbook. Unfortunately I still know way too little about the GNU
autotools to fill in enough material for it to be useful.

Just for fun I include below a very fast written down Q/A sketch. Not
the real thing and the few answers I put in are likely not right.
Don't consider it real questions and answers, I just try show the idea
and what kind of style it could be in.

Maybe there is such a thing already and I just failed to find it.

* General questions

 Q: When do I want to use aufoconf? When automake? When libtool?
    What combinations? Is there ever an idea to use "libtool"
    without the other tools? To use "automake" without "libtool"?

 Q: The "auto" in the name of "automake" and "autoconf", does this
    mean I can create the build environment on one platform and the
    build will work on all Unix/Linux platforms?

 A: Unfortunately not.

    Libtool is kind of a script with collectively built in knowledge
    built up over time, so yes it will in most cases solve the problem
    how to link and create executables and libraries on all platforms.
    It "knows things", as it has its own (a bit unstructured) database
    over what to do. But for a new platform it might be lost.

    Autoconf threoretically will find out what is right by doing
    "tests" instead of relying on any built in knowledge. Tests to try
    find out things about your compiler, linker and operating system.
    And there is a library of macros you can use to customize what
    tests you want it to perform.

    Still, in most cases this is not enough. You will find yourself
    writing new tests for things not covered, and for each new
    platform you start building on, find yourself doing tweaks. If you
    google around you will find most configure.in scripts to be quite
    messy and many of them buggy. So this is not "easy" and definitely
    not "automatic". Automake is more like a toolbox where someone
    else has written many tests saving you time.

    Automake will from a "simplified make file", "Makefile.am", that
    try to look like a specification, create portable make files
    "Makefile.in". When the user unpack he runs the "configure" script
    it will fill in the last parts to create the actual "Makefile".
    Automake works well in most cases. But it is easy to write
    unmaintainable "Makefile.am" files as well.

 Q: If I run into problems and need to call for help, what is the rule
    of thumb for what is to send to <address@hidden,
    <address@hidden> and <address@hidden>?

 A: See the diagram at the page ....., but in general....

    (I'm sure the maintainers doesn't agree, but for me they are
    really one package and I would be happy if they were distributed
    as one package ;-) )

* Cookbook (should be more structured of course)

 Q: How do I make sure "make dist" copies the source I want
    to be included?

 A: All files listed in the varaibles below will be included

      <prog>_SOURCES
      .
      .

    Those in BUILT_SOURCES will....

    Other files in your original source tree that you want
    to be part of the "dist package", you list in  YYY.

 Q: How do I include "almost" all in a directory without listing all files?

 A: You can give a directory to EXTRA_DIST, and then remove what was
    "too much" using as an example

      dist-hook:
            rm -rf `find $(distdir) -type d -name CVS -print`

 Q: What will automatically be cleaned with "make distclean", "make
    clean"...., and how do I add more things to clean away?

 A: There should be "no more" things if you have specified right...
    But you can ....

 Q: What are the different "install" locations and do they differ
    between different flavors of Unix, like Linux and Solaris?

 Q: How do I specify what to install?

    (built, generated, static...)

 Q: What are the rules of thumb what to install where?

 A: Not a GNU autotools question but in general

      $prefix/bin  - ...
      $prefix/sbin - ...
      $prefix/lib  - ...
      $prefix/share/<packagename>  - ...
      .
      .

 Q: Does the install locations follow the Linux LSB specification?

 Q: Can I change the defaults for these standard locations?

 Q: Will it break if I'm over-specifying things to copy or build?

 A: The tools will try figure this out, but remember that this is
    about macro expansions, shell stuff, and at the end will generate
    make files. So try be minimalistic and get it "right" from the
    beginning.

    This varies, but if you are converting from old make files to use
    "automake", for some it might be better to rewrite from scratch
    than to try change the existing files. Clean "Makefile.am" files
    is the key to make them maintainable.

 Q: I have seen some put header files into <prog>_SOURCES, is that
    allowed and when do I want to do that? And when don't I?

 Q: How do I create dependencies between directories in a build?

 A: If using "libtool convenience libraries" it is all taken care
    of (is it?)

 Q: What is this ".libs" directory about?

 A: The reason is (shared library, wrapper, .....)

 Q: But with some objects ending up in ".libs", others not, how can I
    from one directory access objects in another when I don't know
    where automake/libtool will put them?

 A: I general you just have to do things the "libtool way" and then
    you don't need to know the location, use "libtool convenience
    libraries".

 Q: For linking yes, but I have a test script that needs to execute
    things that are part of the build!

 A: You let your script use the script that "knows things", libtool

      libtool --mode=execute <path to exe or exe wrapper above .libs>

 Q: Great! So then I can find the real executable with something like
    "libtool --mode=exepath <path to wrapper or exe>"?

 A: No. The magic to resolve to the real thing is done when it is
    installed. Note that until then it is libtools internal business
    what format the executable of library is in. For a platform it
    might choose to do some more step converting the objects in the
    "make install" phase.

 Q: Shared libraries are to be created with "libtool convenience
    libraries", but I see no shared library target, just the
    ".la" target, why no ".so" target?

 A: Because shared libraries is hard. You even assume all Unix
    operating systems use the ".so" ending, that is not the case.

    This is again the beauty of the "libtool convenience library"
    concept. You "let it do its magic" without you explicity knowing
    the detals, and a "make install" will install the stuff with the
    right extension etc.

 Q: It installs a "<mylib>.la" file as well, how do I prevent that?

 Q: If libtool convenience libraries are so great, when would I ever
    want to specify "lib_LIBRARIES = <name>.a"?

 Q: Can you please take it slow and explain the difference between
    giving "--prefix=<path> to configure, "make prefix=<path>" and
    "make DISTDIR=<path>"?

 Q: I try install a new GNU autotool chain in /usr/local, but
    all gets messed up, what is going on?!

 A: It is a well hidden secret that when installing these tools they
    read each others files and put things in each others directories.
    So the install order is important in some cases. And if you have
    an older toolchain in "/usr" already, a new install in "/usr/local"
    can create interesting effects. To play it safe, start with
    autoconf, then automake and last libtool.

 Q: How can I prevent libtool from compiling source two
    times, when one of the objects will not even be used?

 A: (I don't know .... )

 Q: There is a flag --with-pic, and I can on all 15 platforms I tried
    create static libraries that are position independent using this
    flag, still I hear that this is wrong and will not work, what is
    that about?

 A: (I wish I knew ;-) )

 Q: If I specify --with-pic, isn't the two libtool compilations
    identical, and it still compiles the same source twice?

 Q: I changed GNU autotool version and now things break?! One example
    is that "make distcheck" breaks, but I did no change other than
    installed a new GNU autotool chain!

 A: The GNU autotools have as other software bugs. But the case your
    build files are not compatible with every GNU autotool version is
    not as serious as it sounds. Remember that you are to run these if
    in "maintenance mode", most others that compile from source will
    never run them. Instead they will just run "./configure && make".
    There is no need for "autoconf", "automake" or "libtool" to be
    install on the target build hosts.

    This is the idea. You use tools that have larger requirements on
    the build environment, version etc, and create a source package
    that has less requirements. Other things popular to do in this
    "bootstrap" step is to create yacc/bison output and other
    generated files that will not change depending on build host.
    Saves trouble with failure to build because "you need bison
    version X, sed version Y....".

    For developers in your project the source being picky about GNU
    autotools versions might be a minor but still annoying problem of
    course.

 Q: How do I force use of specific versions of the GNU autotool to
    prevent others from problems using "maintenance mode", when
    using versions I haven't tested or know is broken?

 Q: But I want the source to be ready to compile directly from
    checking out the files from the source repository, no run of GNU
    autotools unless a developer, how do I accomplish that?

 A: (I have heard some rumor some have a sub directory with checked in
    created "libtool" and "configure" scripts, updated "from time to
    time" by the maintainer. That is overrided if you are a developer
    and want to run the autotools yourself. Not sure if this is a
    documented method with some built in support from the tools.)

 Q: Is there help if I want to remove obsolete constructs in
    my "configure.in" files?

 A: Yes, you can give flags to to give warnings or even to rewrite the
    "configure.in" script for you.....

 Q: Should I use "configure.in" or "configure.ac" as the name?

 Q: If "configure.in" is too big, how do I split it up?

 Q: I hate caches, they always mess things up, how do I disable them?

 Q: How do I force a recreation of the "configure" script? Of a
    "Makfile.in"? Of the "libtool" script? Of the ".deps" files?

 Q: So what is the right order to run these separate GNU autotools?

 Q: I'm not a maintainer ordeveloper, and I accidently touched a
    "Makefile.am", and now writing "make" it tries to run some command
    looking for "automake-1.7". I don't have it, how do I convince it
    to stop trying to rebuild things?!

 Q: I'm a developer and love the feature that there are added
    rules that rerun GNU autotools when needed, how do I trigger
    that feature to be added to the "Makefile.in" files?

 Q: What is this "ltmain.sh" about? Should I check it in into
    my repository?

 Q: I'm confused, the same lines about linking seem to be in the
    source of the "libtool" command, the "configure" script and in the
    "ltmain.sh"?! And in "configure" and "libtool" the same text
    repeat itself?!

 A: Is there a question in there? This is just how it is ;-)
    Kidding, "ltmain.sh" was used during the XXX step
    creating the "configure" script. The "configure" script
    is creating the "libtool" command. The duplication is
    because... (yes, why is that?!)

 Q: But if GNU autotools are to run in "maintenance mode", why
    is "libltdl", the portable dynamic loader, tied into the
    libtool package and not separate?

 Q: Argh.. I distributed a TAR source with soft links instead of the
    the actual "libtool" command, why isn't --copy --force the default
    for "automake", "autoconf" and "libtoolize"?

 Q: There is an installed "libtool" command in "/usr/bin", but as I
    understand I run "libtoolize" to create an environment that when I
    run configure creates "libtool" part of my source package, what is
    that "/usr/bin/libtool" about?

 Q: My package use the ".la" file from other packages to find out how
    to link to it, but in some installs this ".la" file is too old or
    broken, in any case not compatible with the current libtool. What
    do I do?

 Q: It is so confusing, some configure script uses "--enable-<feature>",
    others "--with-<feature>" for similar things, and some show in
    the help text 

      --enable-<feature> Enable <feature> (default enabled)

    and let you figure out to use "--disable-<feature>" or
    "--enable-<feature>=no" to negate. Others show in help

      --disabled-<feature> Disabled <feature>

    to show what flag to negate the default. What is the guideline
    here?!

 Q: I find the power of these tools not to be enough, but when looking
    at the generated "Makefile.in" I found some other variables to set
    to enable it to do what I want!

 A: If you stay with the specific version of the tools and you know
    what you do, sure it is possible. But a bad idea.

    If you are really sure the power of the documented interface is
    not enough, I suggest you send a question on the list XXX instead
    of doing an ugly hack. If in luck there is a supported way.

 Q: There are some AM_CFLAGS, AM_CXXFLAGS and AM_LDFLAGS, what are
    they really for? I have seen some use them inside "Makefile.am",
    is that correct usage? So I never do "make CFLAGS=-g" to override?

 Q: I need to from the command line put things before the CFLAGS
    configure figured out, how do I do that? The same question for
    "after"?

 Q: Why is LIBS in the environment propagated into "configure.in", but
    LDFLAGS seems not to be? (Or if it was to the Makefile.in, not
    sure at the moment).

 Q: If using the HP-UX compiler the build need some extra flags in
    CFLAGS and some others in CXXFLAGS, should I let the user specify
    that part of the configure line, or add to my configure script

      case $compiler in
        aCC)
        case $compiler_version in
           1.3*)
              CFLAGS="$CFLAGS ...."
              CXXFLAGS="$CXXFLAGS ...."
              .
              .

 Q: I have in SUBDIRS directory paths like "<subdir>/<subsubdir>", but
    "make dist" doesn't work?!

 A: Automake doesn't like that you jump over a sub directories, put
    a "Makefile.am" in "<subdir>" and let it list the "<subsubdir>".
    

 Q: But really, I need some "guidance" how to approach things, not
    just small usage cases. How do I create a "good" build?

 A: Some advice

   1) If you find a "#ifdef __sun" or other compiler/os specific
      condition in your source it is "bad style". Likely the hard
      coded assumption is already tested by configure, or you can add
      a test. A correct preprocessor tests is on a flag set by
      configure, like "#ifdef HAVE_NAMED_H" or
      "#ifndef HAVE_BROKEN_DLOPEN".

   2) If in "configure.in" you see lots of "case $SYSTEM_TYPE in" you
      might try harder to do a real test instead of assuming things.

   3) Don't do AC_TRY_RUN when you really only need to do
      AC_TRY_COMPILE or AC_TRY_LINK. Makes it easier to cross compile.

   3) For cross compiling to work better, you .....

   4) If you see "for dir in /usr/lib /usr/local/lib" you should...

   5) Some tests will be very hard to read, write lots of
      comments in the configure.in script.

   6) Don't assume "bash" and "gcc" ;-)  [Why do you think Sun C wants -Wall?!]

   7) Don't assume "bash" and "gcc" ;-)  [Do you think those "for" loops are 
standard sh?!]

   8) Don't assume "bash" and "gcc" ;-)  [Why do you assume foo.{c,h} would 
work?!]

 Q: Help! Libtool has ran into some shell limitation and cuts off lines!

 A: If you have bash on the system just not as "/bin/sh", configure
    with "CONFIG_SHELL=/bin/bash". It will set the shell for libtool
    as well.

 Q: I get a complaint the resulting AR archives don't have a symbol
    dictionary, shouldn't libtool make sure ranlib is run?!

 A: You likely fooled libtool. Libtool doesn't test, it has built in
    knowledge and on <some os> it *knows* "ar" will do an implicit
    "ranlib". Now, you likely installed GNU "ar" and put it before the
    <some os> "ar" in PATH, so the needed explicit "ranlib" was never
    called.

 Q: <a lot of questions about convenience libraries, LDADD, LIBADD....>

 Q: <a lot of questions about how to make part of source optional,
    inside the same Makefile.am, directories,....>

Now, others I have spoken to and trust says the current GNU autotools
manuals are crystal clear, so it might just be me that don't get these
tools to the level I want ;-)

And as long as there are such helpful guys as you on this list
I'm not lost without hope when I run into trouble :-)

kent

-- 
Kent Boortz, Senior Software Developer
MySQL AB, www.mysql.com
Office: +46 18 174400 ext. 4450 (VoIP)
Office: +46 19 182931
Mobile: +46 70 2791171