Re: Replacing Guile test-suite with SRFI-64?

[Maxime Devos]

Op 25-09-2023 om 01:32 schreef Mike Gran:
> > A benefit of SRFI-64 is that the SRFI-64 implementation has tests
> > whereas (IIRC) Guile's custom thing doesn't.
>
> OK two separate issues.
> 1. Mike G's gripes about SRFI-64.
> 2. Updating over 65,000 lines of test code

These aren't separate issues -- if SRFI-64 is worse than what Guile has 
currently and SRFI-64 can't be improved to make it better, then that's a 
good reason

> I don't want to be put in the position of having to make a critique of
> SRFI-64, and, once my critique is dissected, make it
> appear that that is justification for updating the test suite.
>
> I wouldn't be the decision maker, anyway.
This last sentence is irrelevant.

> It is 2 MB of test code that has served Guile well so far.

Nobody claimed that these 2MB hasn't served Guile well.  Instead, the 
proposal is about serving Guile better.  That things might be going well 
currently doesn't mean we can't do better.

Also, I dispute that it has served Guile well.  It is difficult to 
select which tests to run (it is easier to delete all the other tests 
than to find/remember the documentation stating how to do this), whereas 
with typical SRFI-64 test runners you can just set 
TESTS=some-scheme-file.scm.  Also test-suite/README is rather obsolete.

A more general problem is that Guile's test library is undocumented.

> I think the argument for srfi-64 is not self-evident. 

Nobody claimed it is self-evident -- instead, Bruno and I gave some 
explanation on how SRFI-64 can be better and didn't say ‘SRFI-64 better 
because duh SRFI-64, self-evident, no?’. (And conversely, you pointed 
out some problems with the current SRFI-64.)

> That
> srfi-64 is familiar to some and has a test suite does not prove
> that the current test suite is difficult for newcomers to understand
> and is buggy.

This message is weird -- you already made a (short) critique about 
SRFI-64, and nobody claimed ‘has test suite -> current test suite is 
difficult to understand / buggy’.  To be clear, this critique is the 
following message.

> IMHO, I think Guile should not depend too heavily on Guile for testing itself.
> It is bad bootstrapping practice.
>
> And personally, I find that errors in srfi-64 tests are more difficult to 
> interpret
> that errors in the guile test suite framework. The logging is worse.

And if it turns out to be justification for updating the test suite, 
then how would that be a bad thing?  After all, it would be justified.

(Also, which test suite are you talking about -- the test suite of 
SRFI-64, or Guile's test suite in its entirety?)

Best regards,
Maxime Devos.

OpenPGP_0x49E3EE22191725EE.asc
Description: OpenPGP public key

OpenPGP_signature
Description: OpenPGP digital signature