Re: GNUism in groff tests, was: pic anomalies

[Ralph Corderoy]

Hi John,

> Wouldn't having a TAP <https://testanything.org/> harness be
> preferable to hand-spun shell-scripts?

I've just read the specification and I don't think it buys us much.
Something still needs to set up the test, run it, echo `ok' for TAP, and
clean up, and something above that needs to run a bunch of tests.  TAP
just seems to summarise ok lines and spot they may have been too few?

> I was rather shocked to learn such a widely-used program as Groff had
> such minimal test coverage.  Testing the output of binary formats is
> understandably difficult, but I think most of those pains can be
> alleviated by testing against Groff's intermediate output format
> first, and then having unit tests assert proper transformation after.

And testing by cmp(1)-ing two canonical PPMs, each containing a page
rendered by Ghostscript, one golden, the other new, would also catch a
lot of regressions for little work, leaving what exactly had caused the
pixels to change as a task for `git bisect'.

It would have caught things like
https://bugs.launchpad.net/ubuntu/+source/groff/+bug/42764

-- 
Cheers, Ralph.