Re: [O] Re: Test framework needed

[MidLifeXis at PerlMonks]

As a heavy Perl user, writing /automated/ tests is a large part of my dev work.

I would suggest / plea / encourage that whatever framework is used can be 
automated.  If it cannot be run as part of an automated process it is not going 
to be run.  Also consider a set of testing platforms (emacs version, supporting 
versions of other .el modules, OS version, external software).  There are many 
dependencies that org has - being able to automate this testing is a must.

Just my $0.02.

Brian / MidLifeXis



----- Original Message ----
From: Eric Abrahamsen <address@hidden>
To: address@hidden
Sent: Wed, March 30, 2011 9:11:23 AM
Subject: [O] Re: Test framework needed

On Wed, Mar 30 2011, Rainer M Krug wrote:

> On 30/03/11 15:46, Eric Abrahamsen wrote:
>> On Wed, Mar 30 2011, Rainer M Krug wrote:
>> 
>>> Hi
>>>
>>> I was bitten again from an unintended regression in org-mode, and that
>>> the second time in two weeks.
>>>
>>> I am probably not the right person to suggest this, but I think it is
>>> time to introduce a test framework for org-mode, to ensure that the
>>> (without doubt useful) approach to develop org-mode does not lead to
>>> regressions.
>> 
>> This would be the page to start with, though the most likely candidate
>> (Elisp Regression Testing) is only available in Emacs trunk at the
>> moment…
>> 
>> http://www.emacswiki.org/emacs/UnitTesting
>
> Am I right in assuming, that all of the possible test frameworks would
> require org files and the expected output (tengle, export to ...,
> agenda, ...)? In this case, would it make sense to start collecting
> those, as they can easily be user contributed, consequently representing
> a cross section of the use cases (even not intended use cases)?

Yup, what you would need is some org source files that exercise all of
the possible export options (for testing export, for example), including
weird edge cases, and then ERT (if that's what we ended up using) would
provide handy functions for making sure the export output matches
expectations. The excellent gentleman who created the ODT exporter,
whose name currently escapes me, has already created test files for his
exporter—that would be a perfect place to start.

Covering all of org's various functions would end up being a bit of a
PITA, though you're quite right that it's an excellent idea, and will
become more and more necessary.

E