Re: Finding the dump (redux)

[Ali Bahrami]

On 4/19/21 8:28 AM, Eli Zaretskii wrote:
> > From: Stefan Monnier <monnier@iro.umontreal.ca>
> > Cc: Eli Zaretskii <eliz@gnu.org>,  emacs-devel@gnu.org
> > Date: Mon, 19 Apr 2021 09:26:57 -0400
> >
> > That reminds me: I think ideally, we should see the pdmp files as the
> > executables and the temacs file as a runtime library (after all,
> > a single temacs file can be used with several different pdmp files).
> >
> > If we could find a nice&portable way to turn the pdmp files into
> > executables, then that would be The Right Way.
>
> I'd rather we didn't mess with binary files and didn't produce
> executables except by running the system linker on object files
> created by some system compiler.  Otherwise we will slip back towards
> the kind of problems we had with unexec.

   I almost floated this yesterday in my reply, but I
dropped it, as things were already getting long, and that
discussion didn't need another digression. However,
I think I see a way to get this data into the binary
itself without doing anything clever with linking, and
only using vanilla C. I saw someone else allude to this
idea here before, so I make no claim to invention, but
let me sketch it out and see if you think it might be viable.

The idea is to get the pdump data into the executable, not
by unexec-like methods, but as a simple C array, compiled
by the C compiler, and linked into emacs like a normal
object. Since we have to build emacs before we generate the
pdump data, this needs 2 stages, much like the temacs/emacs
division as it existed in the unexec days.

Imagine that emacs has an array for this, and a variable
giving the size of the array. A size of 0 means that there
is no pdump content. When emacs starts, it still looks at
the command line option, or for a pdmp file sitting next to
the executable, but failing that, it will fall back to the
array, if present, for its pdump content.

Start with a stub C file containing these variables:

    unsigned char default_pdmp_data[1];
    const size_t  default_pdmp_data_size = 0;

Build temacs with that stub, and then run temacs to
generate a pdmp file, as it does today. temacs will
see that default_pdmp_data[] is empty, and will ignore it.

Then, use a script, or even temacs itself, to generate
a source file that creates a version of default_pdmp_data[]
initialized with the pdmp data from the generated file, and
with a non-zero value for default_pdmp_data_size giving the
size.

And finally, relink emacs once more, using the generated
object in place of the stub.

I notice that there are actually multiple mappings from
todays pdmp file in the process, so the above is probably
over-simplified, and perhaps we need more than one such
array. I'm sure a pdump-knowledgable person would know
what to do about it:

    ali@rtld% pmap `pgrep emacs` | grep pdmp
    00007FE834C00000   7088K rw-----  
/usr/lib/emacs/27.2/x86_64-pc-solaris2.11/emacs-gtk.pdmp
    00007FE8352EC000    120K -------  
/usr/lib/emacs/27.2/x86_64-pc-solaris2.11/emacs-gtk.pdmp
    00007FE83530A000   3188K rw-----  
/usr/lib/emacs/27.2/x86_64-pc-solaris2.11/emacs-gtk.pdmp

(I'm not sure what to make of the mapping with no
access bits set).

I'll guess it also changes the fingerprint, between the
2 stages, but perhaps that's OK, since there's no need
for a fingerprint validation if the data is held in the binary,
or perhaps there's an easy way to finesse that. Perhaps the
fingerprint from temacs gets propagated to the final emacs,
instead of being recomputed?

The overall point of the above is that it only uses
basic C, so it should be robust and portable. Since
we already know that we can read and use the pdump data,
it should work in the same way when seen in an in-memory
array, rather than mapped from a data file.

- Ali