Re: scratch/accurate-warning-pos: Solid progress: the branch now bootstraps.

[Paul Eggert]

On 11/26/18 1:48 AM, Alan Mackenzie wrote:
> > The byte-compiler
> > interpreter would operate more slowly than the normal interpreter, but
> > that's OK. The main and the byte-compiler interpreter could mostly be
> > written with shared code, without slowing down the main interpreter.
> > Admittedly this would not be a project for the fainthearted.
> Indeed not.  Where and how would this help with getting accurate source
> code positions?

From the point of view of the byte-compiler, it should help just as 
well as scratch/accurate-warning-pos does, because it's just another way 
to get the same information. Admittedly it's more work to implement this.


> > any method of outputting source-code locations will founder in the
> > presence of macros.
> scratch/accurate-warning-pos seems to do rather well in this regard.

No doubt it does better than master does. However, macros inevitably 
cause problems when diagnosing problems. If there's a bug in a macro, it 
could well be that the byte compiler will report the line number of the 
call to the macro rather than of the macro itself (where the real bug 
is). So any definition of a "solid" solution must provide some leeway 
for the compiler to issue a "wrong" diagnostic when macros are involved. 
Otherwise, if the goal is to provide a "solid" solution then the 
goalposts will keep moving and there's no way that I could suggest a 
solution that is unambiguously "solid".


> > Instead, attach positions to input objects that are guaranteed to be
> > unique so that retrieval is trivial.
> I think you mean conses here.  I've tried this approach, spending a lot
> of time on it but not getting very far.  The problem is, Lisp objects
> flow through lots of different conses as they are transformed by the
> compiler.  Have a look at cconv-convert, which processes every function.

I took a brief look there and see a lot of calls to mapcar, which of 
course would lose track of locations. But this should be fixable by 
defining and using a new function mapcar-pos, which acts like mapcar 
except it also copies location information from the input list's conses 
to the corresponding conses in the output list.

You're right that this would involve many changes to the byte compiler, 
but the changes should be reasonably mechanical (e.g., change mapcar to 
mapcar-pos) and the overall approach should be preferable to changing 
how 'eq' works.

> The approach I tried
> before to implement this was to ensure that after any source
> transformation, the result was written back to the original cons using
> setcar and setcdr.

If location info is kept in a hash table that maps objects to locations, 
then mapcar-pos can preserve location info merely by updating a hash 
table. There should be no need for setcar and setcdr on the original cons.