Re: GNUism in groff tests, was: pic anomalies

[G. Branden Robinson]

Apologies for some non-specifity regarding the existing tests in the
groff tree at present; I am on vacation and can't comfortably do any
real research at the moment.  I'll be back in the saddle in a week or
so.

At 2019-12-31T13:13:19+1100, John Gardner wrote:
> > As long as these tests use bash(1), i'm very reluctant to do that,
> > even though in general, running a test suite certainly makes sense
> > before you commit a package update.  Given the so far very small
> > test coverage, the tests don't help much for package testing yet.
> > Then again, that is likely to improve in the future, so having them
> > portable would be nice...
> 
> Wouldn't having a TAP <https://testanything.org/> harness be
> preferable to hand-spun shell-scripts?

Possibly.  I happen to be exploring TAP for a work project.  But we need
to consider:

1. The dependency load for those downstream of us.  GNU Bash is about as
ubiquitous as Unix-like software gets, and Ingo/OpenBSD have an allergy
to it.  Few test frameworks can claim greater penetration.  One of those
is POSIX sh.

2. The ease of writing new tests, which should ideally be done by the
person implementing the feature or bug fix.  Shell script knowledge is
almost as ubiquitous as POSIX shells themselves.

I may be biased on this point as I think I've written most of the groff
tests that exist.  It wouldn't be the worst thing in the world if more
people contributed some and drowned me out.  :P

3. What we have is already well-integrated with GNU Automake.  Once
Bertrand showed me the way I took off with the idea of testing.

> Groff already depends on Perl, so it's not as though using prove(1) to
> run tests would add much weight.

Yeah, but would it tighten the versioning of our Perl dependency?

> I was rather shocked to learn such a widely-used program as Groff had
> such minimal test coverage.

Welcome to open source.

I mean free software.

I mean all software, nearly.

The situation has very slightly improved over the past 20 years but only
almost imperceptibly.  Code cowboys don't write unit tests; they bash
together a prototype that is "lean" and "agile" and hurry off to a
different project before anything has to be put into production.  Then
the people who come along behind them try to write unit tests, which (1)
expose the difficulty of understanding "rock star" code (which is
usually shit once you demystify it) and managers perceive code quality
as decreasing because of all the bugs that get uncovered as more
execution traces are exposed.  The junior engineers get treated with
contempt for ruining the codebase that demoed so brilliantly.

A rock star cowboy with a smoke machine to conceal the spit and bailing
wire in his implementation is hailed as a genius.

An engineer who spends effort building a robust system is regarded as a
cost center.

Personally I currently believe formal methods to be the industry's only
hope.

Not that I suggest bringing those techniques to bear on groff.

Yet.  :P

> Testing the output of binary formats is understandably difficult, but
> I think most of those pains can be alleviated by testing against
> Groff's intermediate output format first, and then having unit tests
> assert proper transformation after.

Agreed.  Only the output drivers should need to have final output
formats scraped.  As far as I can imagine, anyway.  I think some of
Bertrand's tests count pages in PS or PDF format (for hdtbl?) but that
may be a matter of convenience.

And who knows--maybe a gditroff page counting utility is something we
should write and add to the distribution.

Regards,
Branden

signature.asc
Description: PGP signature