Re: transitive shared library dependencies and installation

[Roumen Petrov]

Hi Russ, libtool members,


I work on open source projects in my spare time. And so I can not 
provide feedback promptly. Today I have time to re read mail thread and 
to clarify my strong position.

Russ Allbery wrote:
> Roumen Petrov <address@hidden> writes:
>
> > Urgh, article with very limited niche. First is only for shared libraries,
> > next unresolved externals in shared library, and finally requires
> > "advanced" loader.
> Overlinking isn't about unresolved externals in shared libraries.
With unresolved I think about something different but this area is 
mostly out of scope to this thread.

> It's
> about avoiding linking with shared libraries that *don't* resolve any
> external symbols.
This is one point - limited to "shared". Issue is not applicable for 
"static".

>    It doesn't require an advanced loader, just any
> reasonably modern shared library system; Solaris 2.6 from many years ago
> had the same issue.
I'm thinking about Android - with modern shared library system and poor 
loader.
Autotools should work for such hosts as well.

>    It is specific to systems where the linker is willing
> to encode dependencies to libraries (in NEEDED, for instance) that aren't
> actually required, but that's the default behavior of, say, GNU binutils.
Yes binutils adds to "needed" all passed on command line libraries for 
ELF but not for COFF.

> The standard problem is that liba depends on libb, and a binary depends on
> liba (but not libb).
Yes if you mean that binary uses only functions from "liba" and no one 
of all those functions need function from "libb". Let skip use of "data" 
from a library.

If B [binary] uses Fa() [function from a] and Fa() uses Fb() [function 
from] the thinks are different.
If we perform static build we must list both libraries.
Another case is shared build. For COFF is used import library. Import 
library does not add "dependency". So we must list both libraries.

Also for COFF list may contain "extra libraries". Binutil linker will 
link(pick) only ready used - there is no overlinking!


> Libtool's default behavior in the presence of *.la
> files will link the binary with both liba and libb, even though it only
> needs to be linked with liba.
Only in "shared" build for ELF.

> This causes a lot of problems for the
> dependency management in a distribution if allowed to happen archive-wide,
> since it creates tons of unnecessary dependency edges that make dependency
> resolution and archive management much harder.
And the main question is who is responsible to resolve this issues.

> Avoiding *.la files solves most of this.
Using *la files is very helpful for static or for COFF shared builds!

> --as-needed helps when the
> upstream Makefiles are designed to support static linking (where all the
> libraries have to be listed), or where upstream just didn't pay attention
> to this or didn't consider overlinking to be a problem, but has been known
> to have bugs where it leaves out dependencies that actually are required.
This is specific for ELF. It seems to me discussion for linker 
capability exceed scope of this mail list.

> For the software I personally maintain, --enable-reduced-dependencies
> changes the behavior of Autoconf macros to not add the full transitive set
> of libraries and instead only link with the immediate dependencies.  But I
> think I'm the only person who uses that convention.  For newer software
> that uses pkg-config, pkg-config supports this inherently if the library
> maintainers write correct pkg-config files (which a lot of them don't).
>
> The article may have limited applicability in the grand picture of all
> systems and all ways of linking libraries, but it applies to pretty much
> every Linux distribution (and probably any other distribution of any kind
> of UNIX that uses shared libraries and package dependencies).  So by
> quantity of Libtool installs and invocations, it's significant.

Open source is not limited to Linux. Build should be work on many 
OS-es(complilers, linkers, loaders).

Article https://wiki.mageia.org/en/Overlinking_issues_in_packaging 
inform us about issue in preamble.
Unfortunately context is totally misleading. It proposes correction in 
some build helpers - this definitely breaks other platforms.


As exercise:
Write configure script that test for presence of OpenSSL function 
EVP_add_cipher. Use existing macro AC_LINK_IFELSE. Ensure that it will 
work for shared and static build. Ensure that test will work on linux, 
mingw (solaris, android and etc.) hosts.

Note configure script does not use linker wrapper scripts like libtool 
and I would like to suggest do not use macro PKG_CHECK_MODULES.


I guess that after successful implementation my opinion about article 
will be more clear.


Also article point to "underlinking". With other words proposed 
solutions for build helpers (pkg-config, libtool) will lead to 
underlinking on other platforms..


Off topic: For COFF host there is tool that from DLL creates import 
library. DLL lists other needed libraries and COFF does not allow 
"undefined". So for mingw host is easy linker do not "overlink".


"Underlinking" article point that ELF --as-needed is limited.
With other words for autoconf based build we cannot use --as-needed in 
LDFLAGS and "extra library" in LIBS to resolve overlinking and 
underlinking at same time.
Remark: environment variables LDFLAGS and LIBS are autoconf flags and 
there is requirement project to use libtool or pkg-config.


From both articles is clear that ELF overlinking issue remains still 
unresolved. :(
Remark: I agree that is important overlinking to be resolved.


Think also how to resolve overlinking and underlinking if project uses 
plain makefile or non-autotool based build system. This also means that 
only linker is responsible.


Regards,
Roumen Petrov