Re: [MIT-Scheme-devel] [commit 13b5bca] Fix new build to cross from 9.1.1.

[Taylor R Campbell]

   Date: Tue, 15 Oct 2013 11:33:42 -0700
   From: Matt Birkholz <address@hidden>

   > From: Taylor R Campbell <address@hidden>
   > Date: Tue, 15 Oct 2013 02:14:05 +0000
   > 
   > [...]
   >
   > Please back out this change.  The structure I put in place, where the
   > cross-compiler and cross-syntaxer run in the host runtime and not in
   > the target runtime, is absolutely intentional, and necessary to get a
   > sane bootstrap and sane cross-build.

   OK.  What's a "sane cross-build"?  Running a new compiler in an old
   host runtime is "saner" than running a new runtime in an old host
   machine?

Thanks, and sorry for the curt tone -- I was in haste last night and
frustrated that I hadn't explained the rules or motivation for the new
structure very well (which is my fault, not yours).

The toolchain necessary to build the system (macro expander, sf, cref,
compiler) should be able to run in a wide variety of host systems,
including many versions of Scheme (or, ideally, other Schemes like
Scheme48 or Racket), a different host operating system, a different
host machine architecture, a different kind of compiled code, &c.  The
build should furthermore have clear semantics so that we can get
reproducible results in all those different environments.

To do this, we need to isolate what parts of src/ form an essentially
independent Scheme application that we can invoke -- in whatever host
environment we have available -- to build an MIT Scheme distribution.
As an approximation, these parts are currently sf, cref, compiler, and
star-parser.  Star-parser is wrong because it should be another input
to that application, but fixing that is hard.  The macro expander
should be another part, but it's currently integrated into runtime/
and we need to rip it out.

   > The correct fix to the problem you observed is attached.

   That appears to do nothing.

Well, it gets further.

   I cannot build master on 9.1.1 (again).  Is that no longer a priority?

That is still a priority.  It looks like the scaffolding to transition
from a world with SEQUENCE_2 and SEQUENCE_3 to a world with only
SEQUENCE_2 has been dropped, though, which makes this more laborious
than I was hoping it would be...

   I got an "unknown microcode-type: sequence" error, so I tried the
   temporary "training" I put in cref/make.scm before, but the build just
   dies later on with an "Anomalous microcode error unassigned-variable
   -- get a wizard." error while trying to compile optiondb.bin.

   All these errors look familiar, but you did not like my solution,
   so... I await enlightenment.

The anomalous microcode error arises because of the mismatch between
the host's runtime-unx.pkd and runtime.pkg that I described earlier,
because the compiler now uses (SCODE/)SEQUENCE-IMMEDIATE-FIRST, which
did not exist in the host 9.1.1 runtime.

If you replace (scode/sequence-immediate-first ...) by (car
(scode/sequence-actions ...)), and (scode/sequence-immediate-second
...) by (make-sequence (cdr (scode/sequence-actions ...))), it gets
further still.  But I think the that the resulting make.com, dumped by
9.1.1, still has SEQUENCE_3 objects embedded in it, which confuses the
microcode from HEAD.  However, at that point I ran out of time to look
into the problem further last night.

It may be that using my portable fasdumper with the cross-compiler
will suffice to get this working, because it is guaranteed not to dump
SEQUENCE_3 objects even in a host that uses them in memory.

It may be that we could persuade the sf and compiler in-tree to use
their own MAKE-SEQUENCE that is guaranteed to make only SEQUENCE_2
objects even in a host that also has SEQUENCE_3 objects; this might be
less work than working out any issues that may arise with a portable
fasdumper.

However, I am inclined to say that maybe we ought to cut 9.2 from
commit 96ffae5, from before I made a hash of the top-level makefile;
cherry-pick whatever important changes we've made since then; and then
fix up the build (and perhaps get cross-compilation working as a
first-class citizen!) in HEAD for 9.3.  What do you think?