Re: [Gnucap-devel] testing

[al davis]

On Friday 30 January 2015, Felix Salfelder wrote:
> thought so. weird enough, the bug is not present in gnucap-uf
> (i would not have noticed otherwise).

I looked ....  You made some changes there in gnucap-uf (the 
frozen flag), so the bug didn't show in that test case.

Remembering .... very old versions of gnucap and its 
predecessors handled this differently, which didn't necessarily 
lose the probes when expanded again, but it crashed sometimes.  
That was before I was doing such rigorous testing.

That test case just added a resistor, didn't change the node 
map.

So I made another test case that adds a node.  gnucap-uf then 
gave wrong answers because of a mismatch because the subcircuit 
wasn't remapped.  I have some ideas, but for now losing the 
probes is less of a problem than the alternatives.

> to the point. a test that witnesses a bug is valuable and
> should be identifiable. a comment might be sufficient. other
> projects with test suites permit tests that are flagged
> 'expected to fail'. this is a great way to maintain a todo
> list...
> (surprise: this is provided by autotools [1]).

Not really surprising.  autotools does a lot, all mixed up, in a 
very complex way that is impossible to test adequately.