tracepoints/breakpoints and native compilation

[Andy Wingo]

Hi :)

A design question for everyone.  I am wondering how to support
breakpoints, tracepoints, and the like in a Guile with native-code
compilation.  If you are not familiar with what Guile does currently,
see:

  https://www.gnu.org/software/guile/manual/html_node/Traps.html

Basically Guile supports calling out to user-defined procedures when:

  (1) pushing a new continuation (stack frame)
  (2) tail-calling a procedure
  (3) popping a continuation (e.g. on return)
  (4) non-local return (after abort, or after calling a call/cc continuation)
  (5) advancing the instruction pointer

This last one is obviously expensive.  To mitigate this cost, Guile
builds the VM twice: once with hooks and once without.  When run with
--debug, you get the VM with hooks; otherwise, no hooks for you.

Note that without hooks, you still get interrupts, so you can do
statistical profiling, or interrupt a loop.  But unlike hooks,
interrupts are placed in the code.  Practically speaking, without hooks,
what you lose is the ability to set a breakpoint.  You also lose the
ability to trace a procedure, but because tracing can recursively invoke
the right VM, that's not a problem in practice.

Generally you only care about hook performance when they are disabled,
so what you need to do is minimize overhead in that case.  In a bytecode
interpreter (as Guile has now), the hooks do add some overhead, but they
are very predictable branches, so it's not a big deal.  Debug mode is on
by default in the Guile REPL and I think it adds some 10 or 20%
overhead.

However in native-code compilation, hooks are more of a problem.  They
increase the executable size, as each hook invocation has to be present
in the text somewhere.  They pollute the branch predictor, as there are
many more branches.  They increase instruction cache size, even in the
best case where the slow branches are all out of line.  Boo.

So... I have a proposal.  Right now in the short term I am going to make
a JIT for Guile 3.0 (master branch) bytecode.  I am still working on
massaging the bytecode into a state where it is very easily JITtable,
but that will be soon I hope (weeks) and then, outside circumstances
permitting, we can have a JIT within the next 3 months.  For JIT
compilation, the "hook" problem resolves itself very easily: we can JIT
in two modes, one that adds hooks and one that doesn't.  We can alter
the hook API to allow the VM to know when to re-JIT code; adding a
breakpoint doesn't have to re-JIT everything.  Or of course we can keep
the current API and just re-JIT everything.

For the bytecode interpreter, we can keep the two VMs.  Also easy.

However if we ship native code in the .go files -- what do we do?  Three
options, I see -- one, ship "regular" code (no hooks) -- fast, but no
breakpoints from Guile.  Two, ship "debug" code (with hooks).  Or three,
ship the bytecode also, and a JIT compiler, so that we can re-JIT if
needed.

The first possibility would mean that some Guile-compiled code is not
really debuggable in a nice way.  It's not ideal.  The second is OK, but
it would be slow.  However the third option seems to offer a good choice
for general-purpose installs.  If we we ship the bytecode in .go files
by default and the built Guile supports JIT compilation, then maybe we
can get all the advantages -- peak performance with native code without
embedded hook calls, but still the ability to insert hooks if needed.

So I am looking at going with the third option.  It also opens the door
to potential experiments with trace compilation and optimization.  I am
currently not looking at adaptive optimization otherwise, but anyway
that's a bit far off; we need native compilation first to get good
startup time.

Thoughts welcome!

Andy