Re: testing framework and package.el

[Stephen J. Turnbull]

Christian Ohler writes:

 > I see, thanks for clarifying.  Is it really mainly ELisp language 
 > features where this applies?

It applies to everything in the Lispref and User Guide.  We don't have
tests for UI features in most cases (although there is a suite of
tests to make sure the keyboard works correctly, but this is a real
PITA to exercise because it cannot really be done automatically,
except to some extent on X).  I would think Emacs and most packages
would also lack UI tests.

 > worth the trouble.  Still, do you think there's a strict boundary 
 > between shared tests and implementation-specific tests, or a continuum?  

As I wrote before, in my experience, f?boundp is sufficient in most
cases to check for version- or implementation-specific tests.

The implementation-specific tests are unusual, and mostly happen when
functionality is partially moved from Lisp to C or vice-versa,
resulting in the creation of an -internal version of some function.
Once again, (fboundp 'foo-internal) will usually catch that.

 > >   >  a way that these actions are as easy as possible.  I expect that
 > >   >  running the current tests against older Emacs versions is something
 > >   >  that we will want to do much less frequently, so we should not
 > >   >  optimize for this use case at the expense of the others.
 > >
 > > What's so hard about prepending "../" to a few Makefile variables?
 > 
 > I'm not sure I understand your question.

In practice, our tests are either run from within XEmacs with
M-x test-emacs-test-file, or from a Makefile.  Fix those two places
(by prepending "../") and you're golden.

 > I was trying to say that it would be easier to write and maintain
 > tests if the tests for X.el were in X-tests.el in the same
 > directory

That may be true, but in practice I don't know any large applications
that are organized that way.  Do you?