Re: [DotGNU]DotGNU testing standards

[David Sugar]

First, GNU covers a vast territory, and specific testing policies in one 
package may not even be rational in another one.  That being said, it 
may be useful to have some common practices in GNU for testing, and 
actually there are some tools and specific GNU packages to help with 
this like dejangnu.  I think historically testing practices have been 
left to individual maintainers to decide, and while not always perfect, 
the result has been reasonably effective since so many people can and do 
become involved in individual packages over time.

When we are talking about DotGNU, we are of course talking about a 
subset, and in that it may be possible to make some rational and 
reasonable choices.  However, if you require people to submit formal 
regression tests, you may get some to do this, but more likely will get 
far less total reports.  If you ask and suggest this, it may be more 
effective.

Rhys Weatherley wrote:

> "Gopal.V" wrote:
>
> > Hi Guys,
> >        I was having a chat with the official code breaker "Dave
> > -redtalons- Manning". He was enquiring about whether there is
> > any GNU testing standards. I have read the GNU Coding standards,
> > but is there a GNU testing standard. If there isn't one , should
> > DotGNU have one ?.
>
> I'm not aware of a DotGNU testing standard at present.
> It would certainly be helpful to put together a set of
> guidelines for setting up the system, performing tests,
> and what to provide when reporting problems.
>
> I can only speak about pnet, as I'm not familar with the
> bug requirements of the other components.  Right now, unit
> and regression tests are likely to be best for pnet.  There are
> thousand's of little corners in the IL and C# specs, and only a
> regression suite will catch them all.
>
> I've been working on a regression suite for the C# compiler,
> but it still will be several more weeks before it is useful enough
> to release.
>
> The Dave-style "bang the keyword against the wall and see
> if it shatters" type testing is still useful (these tests aren't
> easily reproducible once you run out of keyboards :-) ).
> Such tests can lead to new regression cases to add to the
> main suite.
>
> The question for me would be: should the keyboard-bangers
> be required to submit a regression case, or should we just have
> a formalised list of "provide these things to help someone else
> build a regression case".  I'm open to suggestions.
>
> I will note that Savannah has a bug-tracking system.  It's a
> little primitive for my tastes though:
>
> http://savannah.gnu.org/support/?group_id=353
>
> We could begin by using that system, and then evolve it into
> a better system as we go.  Then it remains for someone to
> compile a list of "provide these things in your bug report".
>
> Cheers,
>
> Rhys.
>
>
> _______________________________________________
> Developers mailing list
> address@hidden
> http://subscribe.dotgnu.org/mailman/listinfo/developers