Re: glibc: ELF_MACHINE_USER_ADDRESS_MASK, sysdeps/mach/hurd/dl-sysdep.c:fmh

[Roland McGrath]

> Hi Roland!

Hi Thomas of the past!  I'm culling old unanswered messages and I
can't always tell which ones are still relevant.  So ignore if not
helpful.

> Richard et moi, we wondered what ELF_MACHINE_USER_ADDRESS_MASK »Mask
> identifying addresses reserved for the user program, where the dynamic
> linker should not map anything.«, the reason for it usage in
> sysdeps/mach/hurd/dl-sysdep.c:__mmap for vm_map's mask argument (but not
> in sysdeps/mach/hurd/mmap.c:__mmap which addmittedly is not »the dynamic
> linker«, so...), and then especially sysdeps/mach/hurd/dl-sysdep.c:fmh
> »XXX loser kludge for vm_map kernel bug« are all about.  You added that
> in commit 5bf62f2d3a8af353fac661b224fc1604d4de51ea (well, waaaay before
> someone would speak about Git commits), and later touched it in commit
> 9ce8b3c817156108b9f1a1cf12a3fa6eb4332f11.

I'll answer in reverse.  

The reason for the "fmh" kludge was that the microkernel's vm_map
"anywhere+mask" logic had a bug.  (IIRC it was something like that in
case with the i386 mask it would start scanning at a high address and
never wrap around and try low addresses, thus falsely failing with
KERN_NO_SPACE all the time.)  At the time (long before "gnumach") it
was far easier to work around microkernel bugs than to rely on their
fixes being in every microkernel you were likely to use--enough so
that it was unremarkable for me to be so lazy as not to even actually
try to fix the microkernel bug (:-().

The reason for using the mask was what the comment says.  The
background is the case of running ld.so directly, as is done in the
libc build for running tests and a few other things.

In the usual case of directly executing an ET_EXEC file, the exec
server loads the ET_EXEC file first, so its segments occupy their
fixed spots in the address space before anything does any "anywhere"
mappings.  Then the exec server maps in the dynamic linker (i.e. the
PT_INTERP file), which is an ET_DYN and so uses "anywhere" mappings.
Then the dynamic linker does some anonymous memory mappings for its
internal malloc, and does some more "anywhere" mappings from files for
the DT_NEEDED objects, etc.

When running ld.so directly, the exec server only maps ld.so itself
and then ld.so starts running.  It needs to do some "anywhere"
mappings of anonymous memory for its internal malloc before it maps in
the ET_EXEC file.  So it's important to ensure that no "anywhere"
mappings it does early on will use parts of the address space that the
ET_EXEC file wants to use.

On Linux the same problem exists in theory.  But it doesn't manifest
in practice because of the kernel's rules for choosing where to place
"anywhere" mappings.  (Depending on kernel vintage and many other
factors, it's either going downwards from high addresses or going
upwards from a fixed starting address that is higher than the
canonical address ranges used by ET_EXEC files that aren't
unreasonably ginormous.)  If it does manifest, it doesn't get noticed,
because "anywhere" mappings in Unix-like monolithic kernels are done
via mmap without MAP_FIXED, which silently destroys old mappings,
whereas Mach's vm_map never does.  So the failure mode is some early
dynamic linker data structures being clobbered, which may well never
cause any harm in a little libc test binary.


Thanks,
Roland