[lmi] Conclusions drawn from certain 2007 xsl-fo experiments

[Greg Chicares]

We need to automate regression testing for pdf output. Some
experimental code from 2007:
  experiment()  in 'emit_ledger.cpp'
  experiment0() in 'ledger_xsl.cpp'
explored various ways of creating pdf output, and saved several
kinds of intermediate files that seemed to be possible candidates
for regression testing. I'm about to remove that experimental
code because the conclusions we can draw from it are now clear.

First, apache fop is written in java, so it's slow. As long as we
use fop at all, it's best to use it only for rendering xsl-fo to
pdf, and use libxslt (fast C) for the
  xml + xslt --> xsl-fo
transformation. We considered xmlroff as an alternative to fop,
and dismissed it:

http://lists.nongnu.org/archive/html/lmi/2006-09/msg00000.html
| we looked at xmlroff and mdc-xsl but they're
| both Unix only

But maybe xmlroff deserves a fresh look:

http://osdir.com/ml/text.xml.xmlroff.general/2005-08/msg00012.html
| Subject: xmlroff works on Cygwin

Second, our best option for regression testing is apparently to
compare xsl-fo files. We considered apache's "area tree", but
found it unsuitable for this purpose:

On 2007-05-30 19:38Z, Evgeniy Tarassov wrote:
> I can't say anything from a 'diff' of Area Tree XML -- even a tiny
> change in cell padding or a column width results in thousands of lines
> being changed, so no matter how many actual changes was done in an xsl
> file, a typical size of a diff file for Area Tree XML would be
> 6000-8000 lines long.

But we can't really say that any one way is best until we figure
out what kinds of defects we want to guard against. Comparing
xsl-fo files makes refactoring safe (given an appropriate set of
testdecks). It also lets us test conditional text, which I guess
is the biggest concern. For layout changes, we'd really need to
compare pdf output, for instance by using these techniques:

http://www.mail-archive.com/address@hidden/msg06094.html
|
| What we do in FOP is calling GhostScript to convert the PDF to a bitmap
| which can then be either put side-to-side with a reference or visually
| "diffed" against a reference. This process isn't automated, yet.

To automate that technique:

http://www.mail-archive.com/address@hidden/msg06098.html
|
| Once you get the PDF to bitmap you can compare the bitmaps
| programmattically.  Get the difference of each channel of each pixel.  A bit
| intensive.
|
| Not only can you tell if there is a difference progammatically, but you can
| also take the result of the pixel differences and produce a "difference"
| bitmap.  If there is no difference it should be a pure black image.

See Figure 7 here:

http://www.menteithconsulting.com/wiki/XTech2007/MyStylesheetRunsBut#xmlroffTestingModule

for a way of viewing differences in "stereo".