[O] Fwd: [test] Mark tests with missing dependencies as "expected to fail"

[Brian Wightman]

Neglected forwarding to the list - sorry Eric for the double post.

Brian


---------- Forwarded message ----------
From: Brian Wightman <address@hidden>
Date: Tue, Oct 18, 2011 at 12:02 PM
Subject: Re: [O] [test] Mark tests with missing dependencies as
"expected to fail"
To: Eric Schulte <address@hidden>


On Tue, Oct 18, 2011 at 11:22 AM, Eric Schulte <address@hidden> wrote:
> I agree it would be preferable to note that not all tests are run when
> dependencies are missing, although I don't think it is extremely
> important.  I think some version of the above would be worthwhile if it
> could be done in a file-wide manner (as are the current dependency
> checks) and wouldn't require duplicating the dependency check or
> changing every test form individually.  Perhaps a file-local-variable
> could be used to expect failures for every form defined in the file?

Perl's TAP* (http://testanything.org/) uses SKIP results for tests
that should not be run because some prerequisite is not available, and
TODO tests for those that are expected to fail (due to not being
implemented, known breakage, etc).  They can be reported separately if
the harness wishes.

It sounds like this is what is being proposed.  Perhaps some prior art
could be used.

* I reference TAP because it is what I am familiar with, not because
it is better or worse than alternatives.

Brian