[Orgmode] Proposal: Emtest as tester

[Tom Breton (Tehom)]

At Carsten's request, I am proposing emtest as the tester for
org-mode.  I would like to hear if there are any objections or
questions.

****** About Emtest

Emtest is an emacs-based test framework.  It reads tests, runs them on
command and presents their results.  Test suites can be run by suite,
by clause, or by library.

It is extensible and modular.  Nearly everything about it can be
replaced or extended.

One important feature is its testhelp libraries:

 * mocks/filebuf - for making mock files and buffers to run tests in.
 * mocks/dirtree - for making mock directory trees.
 * deep-type-checker - for testing that objects, especially
   structures, are type-correct right down to their leaves.
 * match - for pattern-matching.  When you want to test return values
   or similar, but some fields or elements don't have stable values
   (say, a timestamp or a UUID).
 * tagnames - extremely useful for defining test data and iterating
   over examples.
 * testpoint - useful for:
   1. testing functionality that is called deep inside something else,
      where writing a viable test would mean nearly cloning the
      something else to get the calling conditions right.
   2. Testing functionality that uses other functionality that can't
      be easily controlled by passing arguments.
   3. Testing that under given circumstances a certain point is
      reached, not reached, or reached the right number of times.

Also, in less than perfect shape right now:

 * mocks/keystuffer - work in progress, for capturing canned user input
 * misc and standard - standard testhelp functions.  Works but
   undergoing reorganization.
 * types - type specifications, extending what cl provides.  Right
   now, just a few that I needed.
 * persist - useful for tests of inspected output.  Not working right
   now due to redesign of an underlying package.

****** Some questions

  * Where to include it:

    * I'm proposing to put it under org-mode/testing/ So the directory
      structure would look like:

      * org-mode

        * lisp

        * (etc)

        * testing

          * emtest

            * Many files

          * Some support
          * packages emtest
          * uses.

          * org-agenda

            * tests.el

            * (And other test files)

          * org-archive

            * tests.el

          * org-ascii

          * etc (the other org files' directories of test files)

        * (other existing org directories)

  * Should testing of contrib files be in a separate directory?  It's
    not clear to me that it needs to be or should be.

  * Loading.

    Of course this shouldn't require much extra work to build and
    install.  Yet there's a case to be made for not building or
    installing it by default, "them that don't use it doesn't pay a
    cost".

    So I'm thinking I should add another target to the makefile to
    install it, as well as (of course) a test target.

  * How to include it, git-wise.

    What git wants to do with included external projects is to make
    them submodules.  However, I'm told that's a pain to deal with,
    moreso from the other end than from mine.  And it does seem like it
    would be.  Basically git treats a submodule as a single thing, but
    still "signs" it version-wise with a hex ID, and wants it to be the
    correct version.  So git insulates you just a little bit, at the
    cost of having to deal with an additional repository.

    So I'm thinking I'd just include it literally and if that proves
    hard to maintain then we still have the other option.


         Tom Breton (Tehom)