[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Marking bugs as fixed in the test suite
From: |
John W. Eaton |
Subject: |
Marking bugs as fixed in the test suite |
Date: |
Fri, 7 Jul 2017 14:27:39 -0400 |
User-agent: |
Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.8.0 |
About a month ago, I made the following change so that tests for
reported bugs could be tagged as fixed in the test suite by prefixing
the bug number with "*".
http://hg.savannah.gnu.org/hgweb/octave/rev/b6144e6dda9e
With this change, any failing tests that are marked this way are shown
in the test suite summary as regressions. Previously they were just
noted as "expected failures" or "reported bugs" so it was easy to miss
that a change actually reintroduced some bug that had been fixed.
I also added a script and Makefile target ("update-bug-status") that
attempts to automatically tag tests with bug numbers as fixed:
http://hg.savannah.gnu.org/hgweb/octave/rev/0703d31e1ec0
Around the same time, I added bug numbers for as many xtests as I could.
There are just a few left that don't have bug numbers. Any help with
adding bug bug numbers for those would be much appreciated. If bug
reports don't already exist, they should be created. I would have done
it, but I don't really understand the purpose of the remaining xtests.
Maybe someone else could write better reports and I could.
Finally, there are now 4 tests that are always failing and that are
marked as regressions. This is happening because the bug report is
closed as fixed, but there are multiple parts to the report:
http://octave.org/testfailure/?50893
The tests that fail are all items that have been marked as "won't fix".
So, what should we do? Open a separate bug report that won't be closed
as fixed? Add another marker for the tests to indicate "won't fix"?
jwe
- Marking bugs as fixed in the test suite,
John W. Eaton <=