Re: Alternative solution to stat storm problem

[Farid Zakaria]

Hi Ludovic,

On Sat, Jan 8, 2022 at 1:22 PM Ludovic Courtès <ludo@gnu.org> wrote:
>
> Hi Farid,
>
> Farid Zakaria <fmzakari@ucsc.edu> skribis:
>
> > I have written a tool _shrinkwrap_ [2] that takes all transitive
> > dynamic shared object dependencies (only those listed in DT_NEEDED)
> > and turns them into an absolute path.
> >
> > This has the same result as caching the entries and avoids the
> > unnecessary failed attempts at trying each RUNPATH entry.
> >
> > Using the same demo application _emacs_ shows as much as well:
>
> Nice!  I think that’s another interesting way to address the problem.
>
> I guess the advantage is that you don’t need the ld.so patch.  The
> downside is that PatchELF needs to be able to write longer NEEDED
> strings in the dynamic section, which it may not always be successful at
> (I think?).

I can't claim to be a ELF specification guru but I have not
encountered that longer NEEDED strings to be a cause for failure.
The emacs example is a pretty good test case because the transitive
closure of all NEEDED libraries is quite large, which all seem to be
added successfully to the ELF header.

The benefit to me seems:
1 - does not need a glibc patch for functionality (although for other
libc such as musl it might in this case
https://www.openwall.com/lists/musl/2021/12/21/1)
2 - understanding the dependencies of an application become simpler
3 - there are esoteric cases where in fact libraries might link to the
wrong libraries (although they are correct at build time) given a
RUNPATH/RPATH since there are subtleties with the inheritance model.

I'm actually researching ways to improve (3) as well through
mentorship with Tom Scogland by researching alternative ways to do
linking:
- RUNPATH per NEEDED
- the ability to specify whether a RUNPATH should be inherited or not
to downstream dependencies

> Also, I wonder if the absolute file names in NEEDED interfere with uses
> of $LD_LIBRARY_PATH (making it impossible to force use of another
> libxyz.so than the one that would be found in RUNPATH.)

Correct. For a system with reproducibility in mind this can perhaps be
a desired feature.
It is the current limitation of the proposal.

In fact, Carlos brought up a great philosophical question:
"Is linking to libraries through a content-addressable value allowed
for LGPL software?"
What if the linked address also forced the content-address by having
it resolve to something on IPFS ?

> Thoughts?
>
> Thanks for sharing!
>
> Ludo’.