Re: [lmi] Simplify the census-pasting test?

[Greg Chicares]

On 2014-11-15 02:46, Vadim Zeitlin wrote:
> On Sat, 08 Nov 2014 20:14:41 +0000 Greg Chicares <address@hidden> wrote:
> 
> GC> 'wx_test_paste_census.cpp' extracts sample data for pasting from an
> GC> html file that is part of the lmi user manual. Wouldn't it be better
> GC> to write those data inline in that source file? I think that would
> GC> make the code easier to understand, and the test more robust.
> 
>  Definitely.
> 
>  But OTOH it wouldn't test that the values in the user manual can actually
> be pasted and do result in the expected census values (7 columns including
> the "Underwriting Class"). I don't know if this test has any value, but I
> assumed it did, taking into account the original specification description
> of the test. And at least in principle I do see why this could be useful:
> documentation could be forgotten to be updated when something in the
> program is, and the example here could easily get out of date.

There are two possibilities:

(1) The user manual is revised, and a working example is replaced with a
defective example. That should be easy to prevent by testing the change.
The GUI test won't detect that (unless it is also modified to synchronize
it with documentation revisions, but in that case it would be easier just
to test the documentation revisions in isolation).

(2) The underlying code is changed, rendering the user-manual example
nonfunctional. In that case, the GUI test will flag the regression, unless
its data are replaced with others immune to this hypothetical change in
the underlying code. You're correct that this is a real possibility, and
the minimal discipline of testing changed code wouldn't catch it. But I do
strive to work in a more than minimally disciplined fashion; and I have a
good idea of how many end users ever read the manual; I'm pretty sure the
number of people who read this mailing list is vastly larger.

I'm quite sure this won't go wrong, and if it does, that's no disaster.

Taking everything into account, I still prefer to simplify the code by
removing the part that reads this example from the user manual. BTW, we're
treating the examples here:
  http://www.nongnu.org/lmi/sequence_input.html
the same way.