groff
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Groff] Bug in html backend


From: Colin Watson
Subject: Re: [Groff] Bug in html backend
Date: Fri, 16 Aug 2002 14:29:38 +0100
User-agent: Mutt/1.4i

On Fri, Aug 16, 2002 at 02:14:46PM +0300, Mark Veltzer wrote:
> My current idea is to get some pretty offensive manual pages and
> convert them using various back ends. I'll throw in a binary file
> which is not a manual page at all in there too (just to make sure that
> groff is not going into endless loop or crashes). As time goes by and
> as people report bugs it will be the job of the maintainers (you) to
> add files which caused problems (crashes, endless lopps) to groff. A
> directory structure like this will do:
> 
> $groff_root/regression/[man|html|ps|dvi|ascii]

Remember not to hard-code the -man macro set into the design of the test
harness, since there are several others. I think -man is probably one of
the most tested macro sets already, not to mention one of the simpler,
so it will certainly be valuable to test other things as well.

I think a good regression test framework ought to be able to accommodate
not only end-to-end tests like you're proposing, but also minimal test
cases for particular bugs that have been fixed. This is of course a lot
of work, but I'd cite perl as an excellent example of regression tests
that really work and are widely applied. Lots of minimal test cases are
usually more valuable than lots of end-to-end tests in the long run,
since they generally reflect more thought and produce more efficient
coverage.

I'm a little concerned about how to tell whether a test succeeded.
Checking that groff completed successfully is one level, but you also
want to test correct output. On the other hand, you don't want to make
it needlessly painful to make typographical improvements by restricting
at too fine a level what the test framework will accept. I'm not sure
how to solve this. Fortunately many of the things that it would be nice
to test relate to groff's internal computational facilities, and so a
test file could simply use -Tascii and print "ok" or "not ok" based on
the result of its computation; likewise you might often want to test the
exact groff_out(5) output without feeding it through a postprocessor,
which I think will be more stable than something like the output of
grops.

> People wishing to add tests for specific things could just make small
> manual pages with the feature they want to test and add it with the
> expected results to the appropriate directories.

I would expect that most people adding tests for specific things won't
want to use a macro set at all. After all, if you're testing the output
produced by, say, a new input character on various devices, you don't
want to write:

  .SH NAME
  foo \- bar
  .SH TEST
  \(ch

... when you could just write:

  \(ch

... and test that it outputs the right piece of ditroff and that
postprocessors format it into something containing the appropriate piece
of PostScript, HTML, or whatever. Similarly, if you're testing some
feature of pic then you're unlikely to care about what it looks like
when its output is appended to an-old.tmac, fed through troff, and
converted to ASCII - you're only going to be interested in the troff it
produces.

If I were designing this I'd be strongly inclined not to hard-code any
particular testing style into the harness. Instead I'd take advantage of
perl's Test::Harness module, write individual tests as Perl scripts
which output the usual "ok" or "not ok" format, and write a library (to
be extended) of helper functions that an individual test script can use
to describe what command is to be run over the sample input and to help
determine success or failure. These might include "did groff complete
successfully", "does the output match this exactly", "does the output
match this regular expression", "is the output of just running this
preprocessor identical to this piece of troff", and so on. I think the
number of different desirable kinds of test is enough that it's worth
designing in this kind of flexibility, even if you don't use much of it
at first.

(I've picked Perl above because I know it well and I think its test
framework works with an absolute minimum of bureaucratic fluff, but the
exact choice of language doesn't particularly bother me.)

> Can I send the patches in unified diff format ?

I always send patches in that format.

Cheers,

-- 
Colin Watson                                  address@hidden

reply via email to

[Prev in Thread] Current Thread [Next in Thread]