Re: UI tests

[Przemysław Wojnowski]

Richard Stallman <address@hidden> writes:
> Unlike the typical graphical applications, Emacs's UIs are mostly
> controlled by events that have a very shallow translation to Lisp
> data.  Even mouse clicks can be faked, and a convenient Lisp function
> to make that totally easy would not be hard to write.
>
> Perhaps the input side of testing UI features will be fairly easy.
>
> The output side is not so straightforward, but if we set up something
> to accumulate a sort of dribble record of X11 output operations, in
> the form of Lisp-visible data, maybe tests could check that to see
> if they have changed from before.

I'm not sure what do you mean by "output side" here?

My understanding is that inputs and outputs are on system's boundaries
and isolate system's functionality using stable interfaces (common in
different architectural pattern - e.g. [1], [2]).

In Emacs, as you wrote, user actions can be faked and that can be used
in tests.

The other side - the outputs - is not that clear to me. Usually the
outputs are hidden under stable interfaces. The interfaces allow to
exchange implementations without affecting the system and the boundary
is used to test the output of the system. In case of display output,
this won't allow to test every single pixel (so no X11 output), but at
least it will allow to test that the system produces correct output for
display, no matter what that display (its underlying library) is.

Two things to note when introducing automated tests:
1. Tests pyramid [3] - indicates which tests are most important and
ratio between them. TL;DR Emacs should focus on unit tests.

2. Ice Cream Cone anti-pattern [3,4] is a common trap for existing
projects that start to add automated tests.

[1] Layers in "Pattern Oriented Software Architecture vol 1"
[2] http://alistair.cockburn.us/Hexagonal+architecture
[3] 
http://googletesting.blogspot.co.uk/2015/04/just-say-no-to-more-end-to-end-tests.html
(from "The True Value of Tests")
[4] 
http://watirmelon.com/2012/01/31/introducing-the-software-testing-ice-cream-cone/