[gfsd] Re: iozone license free or non-free

[Richard Stallman]

(Janet Casey no longer works for the FSF; I have cc'd gfsd-hackers
instead.)

Anand, would you please send me the full license of iozone?
We need to check for certain that it is a non-free license.

To respond to Capps' points:

    |        A few questions that might help reveal the nature of the
    |    derivative constraint:
    |    a. How many benchmarks exist that permit one to modify the
    |    benchmark and then make comparisons, for publication and
    |    competition with other vendors ?

I think the question does not make sense because of a crucial
ambiguity.

Assuming that people get the benchmark in source code form, nothing
can prevent them from privately changing it and running it.  If they
were to publish the result of running the changed benchmark and
present it as the result of running the unchanged benchmark, that
would be dishonest.  I would not object if THAT were forbidden; and I
suspect that in some cases it would constitute the crime of fraud.

However, if I understand right, the license goes further than
prohibiting just the dishonest publication of the results of running
the changed benchmark; it prohibits changing the benchmark at all.
Is that correct?

If so, the question that Capps asked is not directly relevant.

|    b. Would you trust a benchmark if you knew that your competitors
|    were able to modify the benchmark before they published their
|    results ?

They are certainly able to modify the benchmark -- whether or not the
license permits it.  If they are willing to present dishonest results,
would their consciences stop them from modifying the benchmark because
of a mere license?

The only way we can trust that they did not modify the benchmark is if
either (1) we believe they are too honest to present false results, or
(2) other people can rerun the test using the unmodified benchmark and
present an anomaly in their published results.

The latter possibility does not depend on forbidding modifications.

    |    c. If there were hundreds of different versions of a benchmark,
    |    and they were getting different results, what value would you
    |    place on that benchmark ?

I would pick the benchmark version I thought was best, and compare the
figures of various programs running that version.

    If you are talking about credibility, you can may be have a clause in
    your license that will not allow any modified versions to use your
    name "iozone". Let them call it something else.

All that is needed is to make the name a trademark.

        As for accepting changes and enhancements... I believe
    that there has never been a case when an enhancement
    was not gladly accepted and merged into the official release,
    over the last 16 years. ( Which I suspect is a much better
    track record than Linux code submissions to kernel.org :-)

The question is whether the license of iozone qualifies as a free
software license.  How good a job its maintainer does of installing
patches is simply neither here nor there.

        The credibility issue has risen it's not so pretty face.

I don't know what issue that is.

                                                                 Several
        folks out there have offered up derivative works of Iozone,
        or tried to make their stuff look like a derivative work,
        and given them interesting names like: piozone, xiozone, tiozone,
        and so on.

Is this harmful?  I do not see it.  Those names are clearly different
from iozone.

                   Thus permitting all web searches to find their
        hacked versions, and funky prototypes, that, in some cases, didn't
        even compile !

Is this harmful?  I do not see it.

                       Followed shortly thereafter by me getting tons of
        flaming emails, and a mess of nasty newsgroup postings. I fought like
        crazy to get them to change their names,  to no avail !. It was 
    infuriating !

Why did you want them to change those names?  It is not obvious to me.
And how would society have benefited from such a change?  I don't see.

        Benchmarking is a very special world. Folks out there are involved
        in a very competitive situation and will try to cheat or game
        the benchmark so that their product has an advantage. It's
        not paranoia, its just the world we live in.  Not that long ago
        one of the folks a Samba.org put out a Samba patch to blind
        side Iozone and produce inflated results.

This is something your license has no say about -- right?

So, since you could not threaten to sue them, what method did you use
to convince them to retract the attempt to artificially enhance their
benchmark scores?

        In the world of benchmarking it is critical that the benchmarks
        are not modified. Without this, there is no credibility, comparability
        or a level playing field.

When comparing programs, one needs to know that the benchmarks used
for all the programs were the same, but there are various ways to
achieve that.  I think the non-free license restriction is
unnecessary.

If you are interested, you and I can explore together how to achieve
this goal (useful and clear benchmark comparisons) without making the
benchmark non-free.