[Gcl-devel] Status of test suite

[Paul F. Dietz]

I posted a message about the test suite on the cmucl mailing list
and have been getting some honest, if forceful, feedback.  Some
of the tests are problematic.  In particular, types-4 had problems:
it was requiring base-string to not be a supertype of string,
(for example), but there was nothing in the standard that required
that.  There were other problems as well.  I've fixed those I've
been told about, but I can't be sure there aren't more.

The upshot of all this is that I've been pointedly instructed
to work with the CLOCC test suite rather than do a bad job of
reinventing the wheel.  I can't say I disagree with that; I
don't want to send gcl development down a blind alley.  I may
at some point start offering my tests to be integrated into the
CLOCC suite rather than having a separate test suite in the gcl
tree. I will certainly run the tests against CMUCL to weed out
some bad ones.

What this means for you is to not trust the test suite to
be perfect, or my understanding of the CL spec to be perfect,
since neither are.

I'd feel worse about this if the tests I wrote hadn't already
found two new bugs in CMUCL/SBCL.

        Paul