Re: [lmi] Improving usability of automated tests

[Vadim Zeitlin]

On Wed, 8 Feb 2017 20:56:52 +0000 Greg Chicares <address@hidden> wrote:

GC> I don't yet feel confident about depending on any simple filtering rule.
GC> A test may...
GC>  - succeed entirely
GC>  - run to completion, but report errors
GC>  - start running, but then abend
GC>  - fail to compile
GC> and I need to make sure I'm either
GC>  - detecting absolutely all the abnormalities, or
GC>  - count the successes and compare against an expected total.
GC> But I'm trying not to spend any time on this yet

 I understand this and won't ask you to pay any attention to this right now
but, for future consideration: I've already mentioned "make check" in
Automake-generated makefiles, see
https://www.gnu.org/software/automake/manual/html_node/Log-files-generation-and-test-results-recording.html
for more information.

 And then there is, of course, venerable TAP (https://testanything.org/).
I've never really generated TAP output from C++ and so didn't use any of
the libraries doing it, but it's so simple (which is its main attraction)
that it could easily be produced manually.

 Again, this is just to give you some idea of the existing solutions which
might be useful when/if you return to this question in the future, I
absolutely don't expect anything to be done here now and my personal plan
of action is to make "make check" work and just use it for the observable
future.

 Regards,
VZ