Re: GNUism in groff tests, was: pic anomalies

[Werner LEMBERG]

> A framework helps with none of that.  But a framework can easily be
> so heavy that it distracts from the actual job.  Certainly, we don't
> need a framework without content.

I think the proper way for testing groff would be to make it run with
a fuzzer, using some very simple and small input files.  If a bug gets
found, we have a new testcase (which is automatically stored and used
by the fuzzer for more tests).  Additionally, such a fuzzer framework
also shows the covered code, and by injecting specially crafted test
examples more unused code paths can be activated (and automatically
tested).

Later on, if there is a good code coverage, the available test samples
might be analyzed to check whether they are producing correct output.

Take FreeType as an example; see

  https://github.com/freetype/freetype2-testing/tree/master/fuzzing

(this was a successful GSoC project).  Note that FreeType doesn't have
(yet) tests for checking the validity of its output; right now this is
done by the Chromium project, which also contains a large framework
for checking rendering differences.

Does the Debian project have a fuzzer framework to which groff could
be added?  Or maybe someone could try whether Google is going to
accept groff in the 'OSS-Fuzz' project...


    Werner