Re: Test suite regressions vs expected failures

[Carnë Draug]

On 22 August 2017 at 19:49, John W. Eaton <address@hidden> wrote:
> On 08/22/2017 02:00 PM, Mike Miller wrote:
> [...]
>> We could add another marking like '!' to indicate a bug that shows some
>> behavior that we are not going to fix, but are intentionally adding a
>> test anyway to show that we are not compatible.
>>
>> But since the test will always fail, what does that show? Should the
>> test suite report a problem if a test marked "won't fix" actually passes
>> instead?
>
>
> I'm not sure exactly what to do, but it seems useful to me to somehow note
> incompatibilities that we know about but don't intend to fix so that we have
> some record of them.  That way we have some relatively easy to find pointer
> to the discussion(s) that resulted in marking them as "won't fix".
>

This would be handy for the image package too, I have a few instances
like that.  The test fails, not because I know there's a bug that
needs fixing, but because I want it to fail on purpose.  I guess in
that case a %!error block makes the most sense, since that's the
expected and desired result.

Carnë