Backtrace and enhanced catch

[Neil Jerram]

Neil Jerram <address@hidden> writes:

> I think the only really good fix for this would be to implement an
> exception handling mechanism that doesn't rely on lazy catch, along
> the lines of SRFI-34.

On second thoughts, after rereading SRFI-34, I don't think that's the
solution.  SRFI-34's with-exception-handler is so similar to Guile's
existing lazy-catch that it doesn't give us anything better than we
have already.

I have a solution in mind, however, which is to add an optional fourth
arg ("lazy-handler") to `catch' (and corresponding C APIs, such as
scm_internal_catch), whose effect is the same as what we currently use
lazy-catch for.  For the specific problem that motivated this
investigation - getting a backtrace out of "guile --debug g.scm" - we
can then use this lazy-handler arg to display a backtrace and full
error information.  The following text explains how I've reached this
conclusion.

Overall there are two approaches for capturing or displaying the stack
when an error occurs.  Either you have an enclosing call like
lazy-catch or with-exception-handler somewhere up the call stack,
which specifies a handler to run in the case that an error escapes
back up to that point.  Or you add a throw-hook or error-hook to the
implementation of throw or error, and add the stack handling code to
that hook.  The difference is that the enclosing call approach allows
code inbetween the lazy-catch and the error point to decide on a
different, more local strategy for handling the error, whereas the
hook approach doesn't.  I think it's clear that the enclosing call
approach is better, so will focus on that from here on, but in
principle we could provide both in Guile.  My proposal here doesn't
rule out adding a hook as well in future.

There is a problem, though, with both lazy-catch and
with-exception-handler (which are the current Guile and SRFI-34
implementations of the enclosing call approach).  On the one hand they
require the handler to execute a non-local jump: the lazy-catch doc
says that its handler must not return, and SRFI-34 says that the
handler's continuation is not defined; on the other hand they need to
ensure that this jump is not handled by the same handler again.  In
the SRFI-34 case this leads to a strange rule: the handler must be
executed in the dynamic context of the error, except with the current
with-exception-handler handler removed; and it is not clear what
happens in odd but possible cases where, for example, the jump is to a
location that is still inside the dynamic scope of the relevant
with-exception-handler expression. In the lazy-catch case the
implementation avoids the possibility of the jump being caught by the
same handler by unwinding the dynamic context - all except for the
call stack - back to the lazy-catch expression before it calls the
handler.  So the lazy-catch handler isn't in fact consistently lazy:
fluid values, for example, are not those in the context of the error,
but those back in the context of the lazy-catch expression.

Another (lesser) problem with lazy-catch and with-exception-handler is
that they are always used in practice in a particular pattern.  For
lazy-catch the pattern is

  (catch tag
    (lambda ()
      (lazy-catch tag thunk lazy-handler))
    catch-handler)

For with-exception-handler the pattern (as shown by all the examples
in SRFI-34) is

  (call/cc
    (lambda (k)
      (with-exception-handler
        (lambda (obj)
          ...
          (k 'exception))
        thunk)))

Why is this a problem?  Because it strongly suggests that these forms
are more general than is useful.  And there is a cost to this: a bit
more typing in Scheme, and in the case of Guile a lot more complexity
in the C code needed to set up a catch and lazy catch pair (which is
relevant to the backtrace problem).

We can solve both problems by merging the semantics of catch and
lazy-catch into a single form, an enhanced catch:

 -- Scheme Procedure: catch key thunk handler [lazy-handler]
     
     ... [existing documentation] ...

     If a LAZY-HANDLER is given and THUNK throws an exception that
     matches KEY, Guile calls the LAZY-HANDLER before unwinding the
     dynamic state and invoking the main HANDLER.  LAZY-HANDLER should
     be a procedure with the same signature as HANDLER, that is
     `(lambda (key . args))', and must return normally, in other words
     not call `throw' or a continuation.  It is typically used to save
     the stack at the point where the exception occurred, but can also
     query other parts of the dynamic state at that point, such as
     fluid values.

We would add the same facility to C APIs for setting up catches (with
regard for backward compatibility, of course), and then finally we can
use the modified API to set up a lazy-handler inside
scm_c_with_continuation_barrier() that will save the stack so that a
backtrace can be displayed (or just to display the backtrace directly,
perhaps).

We will also be able to provide a standard lazy-handler for displaying
a backtrace, in both Scheme and C, which I think will give us a nicer
answer to the recurring "how do I get error information" questions on
the mailing list than what we have now.

lazy-catch itself can be deprecated and eventually removed.  All the
lazy-catch uses in Guile's own code follow the pattern above and so
can be rewritten as an enhanced catch, except for one (in
ice-9/emacs.scm) which is incorrect anyway because it can return
normally.

Comments?  If no one objects I will implement this (and fix the
backtrace problem) over the next few weeks.

Regards,
        Neil