libtool-patches
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [patch #9687] bugfix: make -export-dynamic imply --whole-archive


From: Alex Ameen
Subject: Re: [patch #9687] bugfix: make -export-dynamic imply --whole-archive
Date: Sat, 20 Nov 2021 13:21:54 -0600
User-agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.12.0

Thanks for the follow up, this gave me a much clearer idea of the underlying issue that you're trying to solve. I really do appreciate you taking the time to help improve `libtool'.

You're absolutely right that `libtool' completely bungles the use of `--whole-archive' and `--no-whole-archive' which I see as a high priority issue. In several build systems I've built in the past I have needed to apply manual patches to `libtool' to work around this, and I plan on merging changes to fix this soon ( I'm taking my time to create test cases ).

To answer your question about why `--whole-archive' and `--no-whole-archive' are useful flags in general : some code-bases create libraries with multiple builds of objects for differing configurations, which rely the "cherry picking" behavior of the linker in combination with dummy symbols to automatically select the appropriate form on an object for a given link. This is sometimes combined with `--as-needed' and `--no-as-needed' for more fine grained control. These use cases are niche, but they can be incredibly useful - and if I remember correctly I believe that `libpthread.so', GNU `libc.so.6', `libgcc', and several other "core" libraries use this behavior in combination with linker-scripts to "choose the right objects" ( the nitty gritty of how this is accomplished is an interesting rabbit hole to explore ). This reliance on "cherry picking" hopefully sheds light on why `--whole-archive' is not the default behavior, and why those flags exist. Now, having said that I agree that 99.9% of the time when I link a `libfoo.a' into a binary ( especially a shared object ), I usually want the entire archive to link - so `--whole-archive' is a flag I use frequently and it's a major pain that `libtool' doesn't handle it correctly at the moment.

We could got back and forth on whether `-export-dynamic' should imply `--whole-archive' or not, but where I think we absolutely have agreement is that `libtool' needs to start handling `--whole-archive' and `--no-whole-archive' properly. Do you think if those flags were fixed, would invoking `libtool --mode=link -export-dynamic OBJECTS --whole-archive libfoo.a --no-whole-archive libbar.a ...;' be a suitable solution to your issue? Having thought about it more I think my gut concern was basically backwards-compatibility with existing `-export-dynamic' usage.

On your last comment about all of the linking junk being super complex : I totally agree with you, and despite exploring niche `ld' junk for years now I am always learning. I think at bottom this is why I'm cautious about having `-export-dynamic' imply `--whole-archive' automatically; I don't personally know every possible use case for these flags and I would hate to unknowingly break someone's code-base by silently changing the behavior of `-export-dynamic'.


On 11/20/21 10:28 AM, David Lamparter wrote:
Follow-up Comment #2, patch #9687 (project libtool):

Well, it's now 3 years later, and my memory of details on this is pretty much
gone, but let's try.  No warranty on anything here, I'm trying to
reconstruct.

[comment #1 comment #1:]
Could you maybe elaborate on your use case? Perhaps there's something that
I'm missing.

Any executable linked with `-export-dynamic` that links static libraries may
end up missing globally visible symbols if they are not used by the executable
itself.  If the program then tries to load a module that needs these symbols,
it fails.

In my opinion : the existing behavior both by `ld' and `libtool' is
appropriate. Implying `--whole-archive' for dependency libraries in with
`-export-dynamic' will prevent users from intentionally localizing symbols.

`--whole-archive` has no impact on symbol visibility; I don't follow at all
what you're trying to say here.  A binary linked with `-export-dynamic` is
essentially a shared library, and the `ld` docs state:

"This is normally used to turn an archive file into a shared library, forcing
every object to be included in the resulting shared library."

I agree `ld` is doing the appropriate thing here, but `libtool` isn't.  A
binary with `-export-dynamic` needs to export ALL of its global symbols from
ALL of its files for dynamically loaded modules to use.  It is essentially a
shared library at the same time and needs to be treated as such.

NB: `--whole-archive` does NOT make all symbols globally visible! This is
about *objects*, i.e. files.  If you have a file getting linked in that has
some global symbols, but none of them are used in the binary itself - `ld`
will drop the entire file.  That's the wrong thing to do for both creating a
shared library as well as for creating an executable with `-export-dynamic`.

I see these flags as having distinct use cases. I'll note that, the need for
using `--whole-archive' with `ld' isn't necessarily intuitive to users, so I
see the appeal of using it in many situations - but adding additional variance
between `libtool' and `ld' does not seem justified to me.

If I remember correctly, I couldn't find a way to manually add
`--whole-archive` to the linker invocation that libtool does, because the
option is ordering-sensitive to the file names on the ld command line.  I also
do not understand what you mean with "variance" between `libtool` and `ld` -
there is no equivalency here.  `libtool` is a user/wrapper of `ld` that has
the express purpose of figuring out the "right" flags for `ld`, and it's
failing at its job.

Also: what is that distinct use case for `--whole-archive`?  I think it's
exactly this situation right here... the ld docs even hint at it.

If the intention is to export symbols which are defined in statically linked
`libtool' libraries a "convenience library" ( `noinst_' ) might be what you're
actually looking for?

Barring my memory failing me, I think we tried that and it simply has the same
problem.


The patch posted here has been shipped in FRRouting for 3 years now, it has
not caused any breakage in any OS/distribution (prominently: Debian, Ubuntu,
RHEL, *BSD, …) - as such I would urge you to just merge it and move on.  I
would happily help you in understanding the issue here, but I can't do that
with having more or less lost my own understanding of it.  Sorry about that,
but there's a boatload of other stuff I need to understand, and my brain is
not infinite :(.

     _______________________________________________________

Reply to this item at:

   <https://savannah.gnu.org/patch/?9687>

_______________________________________________
   Message sent via Savannah
   https://savannah.gnu.org/




reply via email to

[Prev in Thread] Current Thread [Next in Thread]