bug#67455: (Record source position, etc., in doc strings, and use this in *Help* and backtraces.)

[Alan Mackenzie]

Hello, Stefan.

On Sun, Mar 10, 2024 at 17:03:28 -0400, Stefan Monnier wrote:
> >> >> - Testing `byte-compile-in-progress` can't be right.  Do you to test
> >> >>   whether the result of this backquote will be byte-compiled or do you
> >> >>   really mean to test whether this backquote happens to be executed
> >> >>   during compilation (which could be because the compiler ends up
> >> >>   loading code while executing `eval-when-compile` or `require`)?

> >> > Quite simply, during compilation, all symbols (except nil) get read with
> >> > position, so to strip their positions here would be wrong.

> >> This isn't quite right: during compilation, some code is read with
> >> positions (the code that we will compile), but some code is read in the
> >> normal way (the code we load for the purpose of running).
> >> The distinction is important.

> > OK, I wasn't really counting code that we load as "during compilation",
> > but I take the point.

> The point is that `byte-compile-in-progress` will be non-nil during
> those loads, so you can't use this variable to get the information you need.

Yes.  How about binding it to nil around `load' and recursive edits, and
possibly one or two other things?

> >> More generally: what goes wrong in the above example if you just treat
> >> that as a list of symbol (stripping them all of their position info).
> >> AFAICT when *that* macro is expanded (i.e. ME2) you'll presumably get
> >> code like

> >>     (mapatoms #'(lambda (FOO/p) (DO/p SOME/p (THING/p))))

> >> right?  [ where "/p" means that the symbol has a sympos.  ]
> >> Isn't that sufficient info to add a docstring with position?

> > It's the lambda which has a position rather than the expanded bits from
> > ME2.

> Hmm... then I misunderstand something.  How can the `lambda` have
> a position if you don't include any special treatment of backquote?

In read-positioning-defined-symbols, lambdas get positioned, along with
symbols being defined by defun, defmacro, cl-defmethod, ....  This is not
to do with backquote handling.

> AFAICT the `lambda` in the result of ME1 should not include position
> information because at that time we don't know that it will be used to
> build code rather than be some element of a normal list.

read-positioning-defined-symbols cannot know how (lambda ...) is going to
be used.  It is up to other code (here, macroexp--expand-all) to strip
the position from the lambda when it would be obtrusive.

> And how come the rest doesn't have position information?

read-positioning-defined-symbols is specifically coded (with a state
machine) to position only the lambdas and the defined symbols.

>         Stefan

-- 
Alan Mackenzie (Nuremberg, Germany).