Re: Stopping unit test on failed partial ordering dependency

[Bob Friesenhahn]

On Tue, 23 Apr 2019, Kip Warner wrote:
> Now suppose the TestStartDaemon.sh bails for some reason. There is no
> point on doing all the queries and having them all fail. There has to
> be a sane way of handling this scenario.

Having all the dependent tests fail seems ok, as long as the failure 
is expedient.  Unless the tests are reasonably ordered, it may not be 
obvious to the user why so many tests are failing.

If there are normally 1200 tests run but the user sees only two tests 
run, and one of them fails, then the user might not be aware of the 
many tests which have not been executed.  The tests which fail still 
need to be reported as failed.

One reason why I like the TAP testing framework is that a test script 
has execution context across multiple tests (avoiding repeated 
stop/start), which improves control over the test, and executes more 
efficiently.  You can start your server at the beginning of the test 
script and then execute 500 tests on it via the same script, while 
terminating the server at the end of the script.

Bob
--
Bob Friesenhahn
address@hidden, http://www.simplesystems.org/users/bfriesen/
GraphicsMagick Maintainer,    http://www.GraphicsMagick.org/
Public Key,     http://www.simplesystems.org/users/bfriesen/public-key.txt