bug#38748: 28.0.50; crash on MacOS 10.15.2

[Pip Cet]

On Thu, Jan 9, 2020 at 3:30 AM Eli Zaretskii <eliz@gnu.org> wrote:
> > > No, GC is known to take many thousands of recursive calls to
> > > mark_object.  9000 is not a particularly high number, and doesn't
> > > necessarily signal infinite recursion.
> >
> > In general, you're absolutely correct. But in this case, it still
> > sounds very likely: infinite recursion of a properly tail-recursive
> > function would loop rather than cause a stack overflow, which would
> > explain everything, except for why it's not actually an infinite loop;
> > I suspect the macOS code somewhere does modify things asynchronously.
>
> The backtrace shows a very recursive GC, it doesn't show any other
> function being deeply recursive.  So I'm not sure I understand what
> tail-recursive function did you have in mind.  Can you elaborate?

I can. I think we're looking at two bugs: the first is the simple
use-after-free of XFRAME (frame)->output_data.ns where `frame' is a
dead frame. I've confirmed on GNU/Linux that mark_frame is called for
a frame for which x_free_frame_resources has already been called, if
there's a global variable still referencing the frame. I think the
same thing happens on macOS.

The second one is very tricky, and a hypothesis at best:

1. I think face_inherited_attr is being optimized to tail-call itself
rather than calling itself in a new stack frame; thus, it loops
indefinitely for a faulty face setup which would otherwise lead to an
immediate crash.
1b. that optimization only works without the harmless initialization of "ok".

2. Our initial face setup is faulty in the sense above.

3. Something happens on a secondary thread which causes our face setup
to become non-faulty, possibly during GC.

That would explain the observed behavior, I think, including such
oddities as the bug happening more frequently when running in gdb
(which delays thread creation).