Re: Add a configure option for NATIVE_FULL_AOT?

[Tom Gillespie]

I remember reading the threads where you were working out the
issues using the async approach, and the difficulty of getting
the dumping step to work without issues, so much appreciated.

Short version: I think concerns about NATIVE_FULL_AOT are
orthogonal from issues with having native comp enabled at all.

Long version:

> Maybe for you and some others, but it doesn't mean there isn't a
> conceptual shift.  I've been following the development of the
> native-compilation feature for many months, during which we bumped
> into and resolved quite a few problems with it, and I can tell you
> that some solutions explicitly assumed JIT compilation as the main
> paradigm, otherwise they would have made little sense.

Indeed, however, the presence of the native-lisp/*/preloaded folder
means that the
concerns you raised in the other thread are present regardless of whether
NATIVE_FULL_AOT is set. For example, trying to modify files.el and get those
changes to propagate for an invocation of emacs -q is currently not possible due
to fact that the /preloaded/ els will always be present and will take
priority when
native compilation is enabled (removing files-*.eln from preloaded will also
leave Emacs in a completely broken state as I had the misfortune to learn).

Thus I think those issues are orthogonal to the issue at hand with
NATIVE_FULL_AOT. As Ulrich notes, in the context of a release

> They would live somewhere under /usr where the user has no business.

The issue I ran into was me being an idiot/developer and not something
that users would encounter. That said, you mention the cases where
someone compiles Emacs and installs it locally as their own user. In
those cases they could also shoot themselves in the foot as I did,
but I don't think NATIVE_FULL_AOT has any impact on making it
better or worse. One outstanding question is whether eln-cache
always has priority over native-lisp, but again, that seems like an
edge case.

> several iterations of non-trivial changes to code parts that are quite
> sensitive, because Emacs runs them early during startup, where not
> everything is yet set up.

Looking at the way NATIVE_FULL_AOT is implemented in the make file,
NATIVE_DISABLED is set by default and NATIVE_FULL_AOT disables
that option. Everything after that point works in the same way as the normal
async compilation except that the files are placed in /usr/lib64 (or similar)
instead of in the user eln-cache. The value of native-comp-eln-load-path
includes the /usr/lib64 path by default, so if NATIVE_FULL_AOT is empty
then nothing happens. the preloaded/ directory lives inside of that cach
as well, and that is present regardless of whether NATIVE_FULL_AOT
is set or not.

All of this to say that one of the reasons this never came up is because
there shouldn't be anything special about this (not that there isn't), it shifts
the compilation in time and puts the files further down the search path.

> I didn't invest any thought in it, FWIW.  For me, it was always a "you
> are on your own" option.  I didn't even try it, as all of the efforts
> went into making the JIT compilation working well and reliably.

I think that that will pay off in either case because the underlying
compilation process is identical (if confusingly named). JIT -> async
AOT, and FULL AOT just skipping the async bit for the core and doing
it all at the same time.

Best!
Tom

PS As a point of curiosity, here are the current installed sizes that
I am seeing.

app-editors/emacs-18.59-r14: 413 files, 21 non-files, 6.6M
app-editors/emacs-23.4-r21: 3883 files, 91 non-files, 131.2M
app-editors/emacs-24.5-r11: 4100 files, 97 non-files, 146.9M
app-editors/emacs-25.3-r10: 4234 files, 97 non-files, 150.9M
app-editors/emacs-26.3-r5: 4186 files, 101 non-files, 177.2M
app-editors/emacs-27.2-r3: 4222 files, 102 non-files, 157.3M
app-editors/emacs-28.0.9999-r1: 7292 files, 122 non-files, 304.1M