[Top][All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [Orgmode] Re: Testing --- again...
From: |
Eric Schulte |
Subject: |
Re: [Orgmode] Re: Testing --- again... |
Date: |
Sat, 02 Oct 2010 12:12:37 -0600 |
User-agent: |
Gnus/5.13 (Gnus v5.13) Emacs/23.1.90 (gnu/linux) |
Hi,
This is exciting.
Rather than impose a complete directory/layout schema before-hand I'd
lean towards starting with a little more chaos and then letting the
structure of the test directory develop /naturally/. From the
discussion below it sounds like an initial structure of
testing/lisp/
testing/contrib/lisp/
may make sense, reserving the top level for "meta" testing stuff, like
functions for running tests, common fixtures, example files, etc...
I have two questions.
1) while waiting for ert to be included into Emacs, should we include an
ert distribution as part of the Org-mode repository (maybe using git
sub-modules) or should we just agree that users should have a certain
version of ert installed locally? I'm honestly not sure which of
these options sounds preferable.
2) should the initial population of the testing/ directory take place in
a separate branch of the repository or in the master branch? Again I
don't know which I would prefer, branches add complication but could
result in cleaner commit histories.
Best -- Eric
Carsten Dominik <address@hidden> writes:
> Hi Sebastian,
>
> the lack of a testing suite for Org-mode is really frustrating,
> and even more frustrating is that we have had like seven attempts
> to start one, and each of these lead to nothing. So I would
> be perfectly happy to give a free hand, write access to the repo
> and a full directory in the distribution to implement one.
> Once there is a framework, I am sure many people would be
> willing to contribute tests.
>
> More comments below.
>
> On Oct 2, 2010, at 5:51 AM, Sebastian Rose wrote:
>
>> Hi,
>>
>>
>> I thought about testing again recently. This is something, that never
>> really got started. For a reason: there's no framework for testing.
>>
>> I therefore wrote a very rough proposal, found on
>> http://github.com/SebastianRose/org-test
>>
>> The idea is, to provide two simple commands:
>>
>>
>> * org-test-test-current-defun
>> will search for tests for the defun point is in or behind
>> (`beginning-of-defun') and execute them surrounded by
>>
>> (let ((select (or selector "^org"))
>> (deactivate-mark nil))
>> (save-excursion
>> (save-match-data
>>
>>
>> * org-test-test-buffer-file
>> will search for tests for the entire file and execute them the
>> same
>> way.
>
> FIrst: I have *no* clue about testing.
>
> Second, I am surprised that you want to structure it by function. I
> would have
> thought that it could be structure by file at the most. And then
> there will
> be tests that involve code from many files.
>
> But I guess
>
>>
>> If you use one of these commands, all currently registered ERT tests
>> are
>> deleted, and files are reloaded (since you're likely to work on the
>> tests, too). To repeat the tests without reloading, you will use the
>> ERT commands like `ert-results-rerun-all-tests', bound to `r' in the
>> ERT
>> results buffer.
>>
>>
>>
>> I choose ERT (git clone http://github.com/ohler/ert.git) because
>> that's
>> likely to go into Emacs core (or elpa.gnu.org).
>>
>>
>>
>>
>> The idea is to search the directory structure from the current source
>> file upwards for a directory named "tests/" if it exists. Else ask
>> the
>> user. Similar to what `add-change-log-entry' does.
>>
>> Below that directory, a tree like the source tree exists:
>>
>> project
>> +-- lisp/
>> | +-- a.el
>> | `-- b/
>> | +-- b.el
>> |
>> `-- tests/
>> +-- a.el/
>> | +-- tests.el
>> | `-- a-defun.el
>> `-- b/
>> +-- b.el/
>> +-- tests.el
>> `-- b-defun.el
>>
>> If this setup exists, when editing defun-x in lisp/a.el,
>> `M-x org-test-test-current-defun' will load tests/a.el/defun-x.el
>> (fallback: tests.el there) and execute all tests with selector
>> "^a-defun".
>
> Well, OK, this is fine. But under a.el and b.el there should also be
> general tests that are not function dependent, and there should be a
> place
> to put tests that you do not want to assign to a specific file.
>
> We do have a "testing" directory already, you can use that.
> I would prefer the tests to be in testing, not in lisp/testing
> if possible. I would like to have the lisp directory contain
> only code. If possible.
>
> It would be OK to have a lisp subdirectory in testing,
> just as it would be OK to have contrib/lisp in testing
> for the contributed packages.
>
> PLEASE, go ahead. I do not think you have write access
> yet on repo - give me your user name and I'll activate you.
>
> - Carsten
>
>> `M-x org-test-test-buffer-file' in that same source file will load all
>> *.el files in tests/a.el/ and execute all ERT tests for selector "^a".
>>
>>
>> Thus tests for
>> org-mode/lisp/org-protocol.el
>> will be searched in the directory
>> org-mode/tests/lisp/org-protocol.el/*.el
>>
>>
>> Once the basic route of testing is clear, I'd like to "translate" the
>> existing tests for org-html.el to work with ERT, which will involve
>> writing more tools (create output buffers, compare output with control
>> files using ediff etc.). I know Lennart Borgman has wrote that stuff
>> for nXhtml already. I hope we can use his stuff and help here.
>>
>> The directory org-mode/lisp/tests/ would not need to be part of the
>> "official" Org mode package. It could as well be checked out
>> separately, if "tests" is part of org-mode/lisp/.gitignore (e.g.).
>>
>>
>> Any thoughts?
>>
>>
>>
>>
>> Sebastian
>
>
> _______________________________________________
> Emacs-orgmode mailing list
> Please use `Reply All' to send replies to the list.
> address@hidden
> http://lists.gnu.org/mailman/listinfo/emacs-orgmode
- [Orgmode] Testing --- again..., Sebastian Rose, 2010/10/01
- [Orgmode] Re: Testing --- again..., Carsten Dominik, 2010/10/02
- [Orgmode] Re: Testing --- again..., Sebastian Rose, 2010/10/02
- [Orgmode] Re: Testing --- again..., Sebastian Rose, 2010/10/02
- Re: [Orgmode] Re: Testing --- again...,
Eric Schulte <=
- Re: [Orgmode] Re: Testing --- again..., Sebastian Rose, 2010/10/02
- Re: [Orgmode] Re: Testing --- again..., Eric Schulte, 2010/10/02
- Re: [Orgmode] Re: Testing --- again..., Eric Schulte, 2010/10/02
- Re: [Orgmode] Re: Testing --- again..., Sebastian Rose, 2010/10/03
- Re: [Orgmode] Re: Testing --- again..., Eric Schulte, 2010/10/03
- Re: [Orgmode] Re: Testing --- again..., Sebastian Rose, 2010/10/03
- Re: [Orgmode] Re: Testing --- again..., Eric Schulte, 2010/10/04
- Re: [Orgmode] Re: Testing --- again..., Sebastian Rose, 2010/10/04
- Re: [Orgmode] Re: Testing --- again..., Eric Schulte, 2010/10/05
- [Orgmode] Re: Testing --- again..., Bernt Hansen, 2010/10/05