Re: [DotGNU]Could someone test this on all platforms ?

[Fergus Henderson]

On 09-Feb-2003, Paolo Molaro <address@hidden> wrote:
> for example, the Magnification values need a rationale. If you change
> the magnification you can tweak the final score from pnetmark, this
> doesn't mean that they were tweaked to give a doctored result, but it simply
> means, IMHO, that there is no justification for their current values.
> For example the string benchmark the value is 2771 and for the float one
> it's 4449. This roughtly means you give 1.6x more importance to the float
> benchmark: what is the rationale for this?

I'm not familiar with the details of PNetMark, but if you have
multiple benchmarks, you should use the geometric mean rather
than the arithmetic mean, to avoid this problem.

> A good start
> may be the benchmarks from http://www.bagley.org/~doug/shootout/
> (there is a port to C# of some of the programs at
> http://dada.perl.it/shootout/).

I don't think those would be a good place to start.

> The benchmarks check they were executed
> correctly, there are many more tests, so it should be more
> representative,

Not really, since the tests are all so trivial.

> I would also keep two separate scores for two different kind of
> benchmarks: those that test the execution engine and those that test
> the library implementation.

Most good benchmarks will test both.

-- 
Fergus Henderson <address@hidden>  |  "I have always known that the pursuit
The University of Melbourne         |  of excellence is a lethal habit"
WWW: <http://www.cs.mu.oz.au/~fjh>  |     -- the last words of T. S. Garp.