Re: [DotGNU]DotGNU testing standards

[Rhys Weatherley]

"Gopal.V" wrote:

> Hi Guys,
>         I was having a chat with the official code breaker "Dave
> -redtalons- Manning". He was enquiring about whether there is
> any GNU testing standards. I have read the GNU Coding standards,
> but is there a GNU testing standard. If there isn't one , should
> DotGNU have one ?.

I'm not aware of a DotGNU testing standard at present.
It would certainly be helpful to put together a set of
guidelines for setting up the system, performing tests,
and what to provide when reporting problems.

I can only speak about pnet, as I'm not familar with the
bug requirements of the other components.  Right now, unit
and regression tests are likely to be best for pnet.  There are
thousand's of little corners in the IL and C# specs, and only a
regression suite will catch them all.

I've been working on a regression suite for the C# compiler,
but it still will be several more weeks before it is useful enough
to release.

The Dave-style "bang the keyword against the wall and see
if it shatters" type testing is still useful (these tests aren't
easily reproducible once you run out of keyboards :-) ).
Such tests can lead to new regression cases to add to the
main suite.

The question for me would be: should the keyboard-bangers
be required to submit a regression case, or should we just have
a formalised list of "provide these things to help someone else
build a regression case".  I'm open to suggestions.

I will note that Savannah has a bug-tracking system.  It's a
little primitive for my tastes though:

http://savannah.gnu.org/support/?group_id=353

We could begin by using that system, and then evolve it into
a better system as we go.  Then it remains for someone to
compile a list of "provide these things in your bug report".

Cheers,

Rhys.