Re: Speeding up the bootstrap build - a quick hack.

[Alan Mackenzie]

Hello, Eli.

On Wed, Jan 19, 2022 at 19:03:46 +0200, Eli Zaretskii wrote:
> > Date: Wed, 19 Jan 2022 16:50:11 +0000
> > Cc: monnier@iro.umontreal.ca, emacs-devel@gnu.org
> > From: Alan Mackenzie <acm@muc.de>

> > The idea is to leave lisp/Makefile.in unchanged, and make all the
> > alterations in src/Makefile.in.

> > The segment of code you cite above builds all of compile-first in one
> > invocation, so there's no need to worry about mixtures of interpreted
> > source and .elc.

> > So, we duplicate that bit of code, setting the Emacs variable
> > no-native-compile on the command line of the first occurrence.  This
> > will cause the byte compilation of all of compile-first.

> > After this bit of new code, we use 'touch' to set the date of these new
> > *.elc's back to the distant past.  This will ensure that these *.el's
> > get built again in the next step.

> > The second duplicate of the old bit of make code will build the .eln's,
> > using the ("very old") .elc's which are still available inside Emacs.
> > It should do this reasonably quickly, because it is using *.elc's.

> Yes, that's the idea.

> > Yes, I agree that this approach is more elegant and surely easier to
> > maintain than my original hack, if it can be made to work (which it
> > surely can).  I will look at this this evening.  Thanks for the
> > suggestion.

> Thank you.

OK, things didn't go quite according to plan.  The name compile-first
only really has meaning in lisp/Makefile.in, so most of the change had to
happen there.

What gave trouble was the setting of BYTE_COMPILE_FLAGS.  It contains
(setq load-prefer-newer t), which prevents the new mechanism from
working; the newly build seven .elc files have been given a date in 1970,
so the source files are newer and thus get loaded in place of the .elc's.

This was surprisingly difficult to solve.  There appears to be no way in
make to set a variable depending on what the target is.  The make manual
doesn't say this explicitly, it just depends on vagueness.  After an hour
of searching for such a feature, it gradually dawns on you that there is
no such feature, even though one might be expected.  I'm glad the Emacs
manuals aren't like that.

So I "solved" this problem by passing an argument as an environment
variable from src/Makefile.in to lisp/Makefile.in.  Yuck!  If anybody can
come up with a better way, it would be appreciated.

Anyhow, this new scheme appears to work, and it's almost certainly better
than the first approach.  Here's the current patch.  Comments and
criticisms would be welcome.

Thanks!



diff --git a/lisp/Makefile.in b/lisp/Makefile.in
index 3a72034463..8f4001cf24 100644
--- a/lisp/Makefile.in
+++ b/lisp/Makefile.in
@@ -74,9 +74,13 @@ loaddefs =
 AUTOGENEL = ${loaddefs} ${srcdir}/cus-load.el ${srcdir}/finder-inf.el \
   ${srcdir}/subdirs.el ${srcdir}/eshell/esh-groups.el
 
+ifeq ($(PREFER_ELC),yes)
+BYTE_COMPILE_FLAGS = $(BYTE_COMPILE_EXTRA_FLAGS)
+else
 # Set load-prefer-newer for the benefit of the non-bootstrappers.
 BYTE_COMPILE_FLAGS = \
   --eval '(setq load-prefer-newer t)' $(BYTE_COMPILE_EXTRA_FLAGS)
+endif
 
 # Files to compile before others during a bootstrap.  This is done to
 # speed up the bootstrap process.  They're ordered by size, so we use
@@ -303,9 +307,16 @@ .SUFFIXES:
 # An old-fashioned suffix rule, which, according to the GNU Make manual,
 # cannot have prerequisites.
 ifeq ($(HAVE_NATIVE_COMP),yes)
+ifeq ($(ANCIENT),yes)
+.el.elc:
+       $(AM_V_ELC)$(emacs) $(BYTE_COMPILE_FLAGS) \
+       -l comp -f batch-byte-compile $<
+       touch -t 197001010000 $@
+else
 .el.elc:
        $(AM_V_ELC)$(emacs) $(BYTE_COMPILE_FLAGS) \
        -l comp -f batch-byte+native-compile $<
+endif
 else
 .el.elc:
        $(AM_V_ELC)$(emacs) $(BYTE_COMPILE_FLAGS) -f batch-byte-compile $<
diff --git a/src/Makefile.in b/src/Makefile.in
index 04fabd5f42..4b28c01f03 100644
--- a/src/Makefile.in
+++ b/src/Makefile.in
@@ -914,7 +914,11 @@ $(bootstrap_pdmp):
        $(RUN_TEMACS) --batch $(BUILD_DETAILS) -l loadup --temacs=pbootstrap \
                --bin-dest $(BIN_DESTDIR) --eln-dest $(ELN_DESTDIR)
        @: Compile some files earlier to speed up further compilation.
-       $(MAKE) -C ../lisp compile-first EMACS="$(bootstrap_exe)"
+       @: First, byte compile these files, ....
+       ANCIENT=yes $(MAKE) -C ../lisp compile-first \
+       EMACS="$(bootstrap_exe)"
+       @: ... then use them in native compiling these and other files.
+       PREFER_ELC=yes $(MAKE) -C ../lisp compile-first EMACS="$(bootstrap_exe)"
 endif
 
 ### Flymake support (for C only)
diff --git a/src/verbose.mk.in b/src/verbose.mk.in
index e3f5678303..01076df946 100644
--- a/src/verbose.mk.in
+++ b/src/verbose.mk.in
@@ -40,12 +40,17 @@ AM_V_CXX     = @$(info $   CXX      $@)
 AM_V_CCLD    = @$(info $   CCLD     $@)
 AM_V_CXXLD   = @$(info $   CXXLD    $@)
 ifeq ($(HAVE_NATIVE_COMP),yes)
-ifeq ($(NATIVE_DISABLED),1)
+ifneq ($(NATIVE_DISABLED),1)
+ifneq ($(ANCIENT),yes)
+AM_V_ELC     = @$(info $   ELC+ELN  $@)
+AM_V_ELN     = @$(info $   ELN      $@)
+else
 AM_V_ELC     = @$(info $   ELC      $@)
 AM_V_ELN =
+endif
 else
-AM_V_ELC     = @$(info $   ELC+ELN  $@)
-AM_V_ELN     = @$(info $   ELN      $@)
+AM_V_ELC     = @$(info $   ELC      $@)
+AM_V_ELN =
 endif
 else
 AM_V_ELC     = @$(info $   ELC      $@)


-- 
Alan Mackenzie (Nuremberg, Germany).