bug#65051: internal_equal manipulates symbols with position without checking symbols-with-pos-enabled.

[Alan Mackenzie]

Hello, Stefan.

I haven't changed my view that the current handling of SWPs in equal is
a bug, and that the bug should be fixed.  Your patch isn't this fix.

I continue to be uneasy about the contradictions in your attidude where,
on the one hand, you say "anything we do here sucks equally" and
describe this conversation as "bikeshedding", and on the other hand
you're steamrolling over my clear vision for fixing the bug.

Nevertheless, here are my comments on your proposed patch, as promised.


On Sat, Aug 12, 2023 at 14:46:24 -0400, Stefan Monnier wrote:
> Thanks,


>         Stefan


> diff --git a/doc/lispref/symbols.texi b/doc/lispref/symbols.texi
> index 34db0caf3a8..d2e397faf80 100644
> --- a/doc/lispref/symbols.texi
> +++ b/doc/lispref/symbols.texi
> @@ -782,11 +782,16 @@ Symbols with Position
>  @cindex symbol with position
 
>  @cindex bare symbol
> -A @dfn{symbol with position} is a symbol, the @dfn{bare symbol},
> -together with an unsigned integer called the @dfn{position}.  These
> -objects are intended for use by the byte compiler, which records in
> -them the position of each symbol occurrence and uses those positions
> -in warning and error messages.
> +A @dfn{symbol with position} is a pair of a symbol, the @dfn{bare
> +symbol}, together with an unsigned integer called the @dfn{position}.
> +Symbol with position cannot themselves have entries in obarrays
> +(contrary to their bare symbols; @pxref{Creating Symbols}).

"Cannot" is inappropriate here, since there is nothing regrettable about
SWPs not being stored in obarrays.  "Contrary" is also inappropriate
since there is no disagreement, opposition, or conflict between the two
things.  "As contrasted with" would be better.  On the other hand, why
didn't you just leave it as I had left it?

> +
> +Symbols with position are for the use of the byte compiler, which
> +records in them the position of each symbol occurrence and uses those
> +positions in warning and error messages.  They shouldn't normally be
> +used otherwise.  Doing so can cause unexpected results with basic
> +Emacs functions such as @code{eq} and @code{equal}.
 
>  The printed representation of a symbol with position uses the hash
>  notation outlined in @ref{Printed Representation}.  It looks like
> @@ -798,11 +803,21 @@ Symbols with Position
 
>  For most purposes, when the flag variable
>  @code{symbols-with-pos-enabled} is non-@code{nil}, symbols with
> -positions behave just as bare symbols do.  For example, @samp{(eq
> -#<symbol foo at 12345> foo)} has a value @code{t} when that variable
> -is set (but @code{nil} when it isn't set).  Most of the time in Emacs this
> -variable is @code{nil}, but the byte compiler binds it to @code{t}
> -when it runs.
> +position behave just as their bare symbols would.  For example,
> +@samp{(eq #<symbol foo at 12345> foo)} has a value @code{t} when the
> +variable is set; likewise, @code{equal} will treat a symbol with
> +position argument as its bare symbol.

This is suboptimal for your version.  The paragraph is about Emacs's
behaviour when symbols-with-pos-enabled is non-nil.  You're including
behaviour in this paragraph which you want to be independent of
s-w-p-enabled.  It really needs its own paragraph.

> +
> +When @code{symbols-with-pos-enabled} is @code{nil}, any symbols with
> +position continue to exist, but do not always behave as symbols.

The "do not always" is vaguer than it should be.  The doc should be
explicit about when SWPs behave as symbols, and when not.

> +Most importantly @code{eq} only returns @code{t} when given truly
> +identical arguments, for performance reasons.  ....

There's more to it than "for performance reasons".  I think you could
omit ", for performance reasons" since it is more likely to cause
confusion than anything else.

>                                            .... @code{equal} on the
> +other hand continues to treat a symbol with
> +position argument as its bare symbol.

"Continues to" is inappropriate here, since there is nothing continuous
happening, nor a continuous "treating" of anything.  A useful phrase
might be "regardless of the value of @code{symbols-with-pos-enabled}".
But as noted above for s-w-p-e's non-nil case, this doesn't belong in
the paragraph about the behaviour when s-w-p-enabled is nil.

> +
> +Most of the time in Emacs @code{symbols-with-pos-enabled} is
> +@code{nil}, but the byte compiler and the native compiler bind it to
> +@code{t} when they run.
 
>  Typically, symbols with position are created by the byte compiler
>  calling the reader function @code{read-positioning-symbols}

What's missing:  (i) You'll need to amend the definition of
symbols-with-pos-enabled in its @defvar, since you're changing it's
meaning.  (ii) You should document the behaviour of SWP's in the
sections on `eq' and `equal'.  (iii) See below.

> diff --git a/src/fns.c b/src/fns.c
> index ac30670b3ac..fde4ef6b08b 100644
> --- a/src/fns.c
> +++ b/src/fns.c
> @@ -5166,7 +5166,7 @@ sxhash_obj (Lisp_Object obj, int depth)
>           hash = sxhash_combine (hash, sxhash_obj (XOVERLAY (obj)->plist, 
> depth));
>           return SXHASH_REDUCE (hash);
>         }
> -     else if (symbols_with_pos_enabled && pvec_type == PVEC_SYMBOL_WITH_POS)
> +     else if (pvec_type == PVEC_SYMBOL_WITH_POS)
>         return sxhash_obj (XSYMBOL_WITH_POS (obj)->sym, depth + 1);
>       else
>         /* Others are 'equal' if they are 'eq', so take their
  
Have you tested this thoroughly?  Hash tables were one of the more
troublesome things to get right when I was developing this stuff.  It
also needs documenting in the hash table chapter of the elisp manual.

-- 
Alan Mackenzie (Nuremberg, Germany).