bug#66759: 30.0.50; Flymake (with Eglot) error cleaning up old overlay

[Richard Copley]

I'm afraid I'm unable to consistently reproduce this error. I hope you
can see the issue and devise a testcase from the following
description.

The function `flymake--publish-diagnostics' runs in two phases.
In phase 1 it deletes all the overlays referenced by the (now stale)
existing diagnostics. In phase 2 it creates new overlays for the newly
published diagnostics.

Phase 1 invokes `overlay-buffer' on the value of the :overlay property
of each diagnostic. This signals `wrong-type-argument' if the value is
nil.

Phase 2 intends to set the :overlay property of each diagnostic by
calling `flymake--highlight-line'.

The function `flymake--highlight-line' may return without setting the
:overlay property. For example:

    (when (= (overlay-start ov) (overlay-end ov))
      ;; Some backends report diagnostics with invalid bounds.  Don't
      ;; bother.
      (delete-overlay ov)
      (debug) ;; -- SEE BELOW (last line)
      (cl-return-from flymake--highlight-line nil))

This is likely to occur when `flymake-start` is called by the idle
timer, if there was a diagnostic near the end of the file and the user
has just made the file shorter. In that case, phase 2 inserts a
diagnostic with a null overlay into `flymake--state-diags'.

When `flymake--stated-diags' contains a diagnostic with a null
overlay, the next call to `flymake--publish-diagnostics' signals
`wrong-type-argument' during phase 1. If this next call also has
`flymake-start' in its call stack, the signal is caught in
`flymake--run-backend', which disables the backend.

If the backend continues to call its reporting function (as does Eglot
whenever the language server publishes diagnostics), this leads to a
flood of "Unexpected report from disabled backend %s" errors and an
unusable Emacs session.

A possible fix is to check if `flymake--highlight-line' created an
overlay before inserting a diagnostic into `flymake--state-diags',
in phase 2.

Another is to check that the :overlay property is not null before
attempting to examine and delete the overlay, in phase 1.

Attached is a backtrace from a real-life occurrence of the issue,
obtained by inserting `(debug)' (SEE ABOVE (quoted code)).

backtrace.txt
Description: Text document