bug#59317: 29.0.50; Feature idea: suppress `message' output in ert batch test output

[Matt Armstrong]

Stefan Kangas <stefankangas@gmail.com> writes:

> Eli Zaretskii <eliz@gnu.org> writes:
>
>>> A good number of Emacs tests exercise parts of Emacs that call
>>> `message'.
>>
>> Are you sure?  Many Emacs features display text that doesn't go
>> through 'message'.
>>
>>> Idea: hide `message' output when running in batch mode, printing them
>>> only for failed tests.
>>
>> We have set-message-function that could be used for this purpose, I
>> think?  If indeed 'message' is the culprit.
>
> Is there any way to capture and stop *all* output in the terminal for
> passing tests?  Like `set-batch-output-function', or something to that
> effect.

The call sequence to look at is:

message3_nolog:xdisp.c ->
   message_to_stderr:xdisp.c ->
   errwrite:sysdep.c

There is errstream:sysdep.c, which is where any temporary redirection of
error output could go, but there is no existing way to change it
temporarily from lisp or even C.  Further, we'd probably want to save
the output somewhere in case the test did fail (or we trust the
*Message* buffer...not sure if that is there in batch mode?).

A concern I have is suppressing useful diagnostic errors.  E.g. at any
point emacs_abort() can run kill-emacs hooks which can call `message'.
Is it worth the complexity of adding temporary output redirections, to
clean up test output, at the risk of making this kind of thing harder to
debug?

A simple approach is to run each test in an isolated subprocess, but
that would be much slower.