bug#66782: 29.1; ERT tests report test redefined depending on loading sequence

[Xiyue Deng]

Mattias Engdegård <mattias.engdegard@gmail.com> writes:

> 28 okt. 2023 kl. 23.19 skrev Xiyue Deng <manphiz@gmail.com>:
>
>> To clarify one of my
>> original proposals: if it makes no sense for a `ert-deftest' to depend
>> on other `ert-deftest', I would say there is no reason for such a
>> dependency, and people should move reusable components to a helper
>> module instead.
>
> There is no reason for one `ert-deftest` to make use of another, but was that
> the case here? I interpreted it as a test suite that spread its `ert-deftest`
> definitions over multiple .el files which isn't necessarily a bad way of
> organising them.
>

As they just use eask to run all tests under folder test in their
Makefile[1], I don't think spreading definitions is what they are doing.
To clarify, I think instead they are requiring lsp-integration-test just
because they want use the `lsp-test-wait' macro[2] in
lsp-clangd-test[3][4][5], which could have been done by putting those
macros in a separate file and requiring that from both tests.

> As said before, there is little sense in including the same individual tests 
> from multiple 'root' .el files since that would run them more than once.
>
> Perhaps that is what you meant? If so, please forgive me for the pedantry.
>

Exactly, and no worries.

As far as I know, there is no practical use for a test case to depend on
another test case in any language I worked with as a test case provides
no reusable component.  On the other hand, there are alternatives that
makes more sense to do what they (e.g. lsp-mode) are doing.  Again, IMHO
I think it makes sense for ERT not to support it or to even forbid it.
Doing this also avoids such misleading error message I was reporting,
which got triggered depending on the loading sequence which may make
test suites flaky (as the loading sequence is not deterministic).

[1] https://github.com/emacs-lsp/lsp-mode/blob/master/Makefile#L42
[2] 
https://github.com/emacs-lsp/lsp-mode/blob/master/test/lsp-integration-test.el#L51
[3] 
https://github.com/emacs-lsp/lsp-mode/blob/master/test/lsp-clangd-test.el#L109
[4] 
https://github.com/emacs-lsp/lsp-mode/blob/master/test/lsp-clangd-test.el#L118
[5] 
https://github.com/emacs-lsp/lsp-mode/blob/master/test/lsp-clangd-test.el#L141

-- 
Xiyue Deng