Re: [Gnucap-devel] testing notes on the wiki

[al davis]

On Saturday 14 September 2013, al davis wrote:
> On Friday 13 September 2013, Felix Salfelder wrote:
> > I'd like to give it a try. where are the tests from "3. Run
> > gnucap on the test input files."?
> 
> It will be in git shortly.

It's there now.

tests directory in master.

In the past, I distributed the tests in a separate tarball.  In 
some cases, from snapshot to snapshot there were no changes to 
tests ("yet") so I didn't make another.  That is why the date on 
the tarball I referenced is not the latest.

A few reasons for the separate tarball ...

One is that some of the tests are designed to test not just for 
bugs but for things like roundoff error.  Some of these can 
produce different results with different compile options, 
different operating systems, different CPUs.  The difference is 
not significant (shouldn't be) but does show with a "diff".  In 
cases like this, providing the tests to casual users would lead 
them to believe something is wrong, when it isn't.

Some files are old enough that they were substantially complete 
before the formal procedure was in place.  That is where you see 
"untested" in released code.  To show the effectiveness of the 
system, ALL (100%) of the bug reports ever received went into 
either "untested()" code or into a module where the procedure 
was not applied.