bug#43004: 28.0.50; Test failures due to symlinked Emacs sources

[Stephen Berman]

After my latest build from master I ran make check (for the first time
in a long time) and got the following 20 unexpected failures:

In test/lisp/progmodes/elisp-mode-tests.el 14 unexpected results:
   FAILED  xref-elisp-test-find-defs-defgeneric-co-located-default
   FAILED  xref-elisp-test-find-defs-defgeneric-el
   FAILED  xref-elisp-test-find-defs-defgeneric-implicit-generic
   FAILED  xref-elisp-test-find-defs-defgeneric-no-default
   FAILED  xref-elisp-test-find-defs-defgeneric-no-methods
   FAILED  xref-elisp-test-find-defs-defgeneric-separate-default
   FAILED  xref-elisp-test-find-defs-define-overload-co-located-default
   FAILED  xref-elisp-test-find-defs-define-overload-no-default
   FAILED  xref-elisp-test-find-defs-define-overload-no-methods
   FAILED  xref-elisp-test-find-defs-define-overload-separate-default
   FAILED  xref-elisp-test-find-defs-defun-defvar-el
   FAILED  xref-elisp-test-find-defs-defun-el
   FAILED  xref-elisp-test-find-defs-defvar-el
   FAILED  xref-elisp-test-find-defs-feature-el

In test/lisp/help-fns-tests.el 5 unexpected results:
   FAILED  help-fns-test-alias-to-defun
   FAILED  help-fns-test-bug23887
   FAILED  help-fns-test-lisp-defsubst
   FAILED  help-fns-test-lisp-defun
   FAILED  help-fns-test-lisp-macro

In test/lisp/emacs-lisp/cl-generic-tests.el 1 unexpected result:
   FAILED  cl-generic-tests--method-files--finds-methods

It seems that the failures arise because my Emacs sources are in a
directory on a different partition from my home directory, and the
source directory is symlinked from §HOME, and my Emacs build directory
is under $HOME.  When I do batch runs of the test three files in
question starting from my home directory (i.e. dereferencing the symlink
to the sources), I get the the failures I reported; but when I start the
batch runs from the real directory containing the test sources, the one
test in cl-generic-tests.el that failed now passes, and of the fourteen
xref tests in elisp-mode-tests.el that failed, now only these five fail:

   FAILED  xref-elisp-test-find-defs-defgeneric-el
   FAILED  xref-elisp-test-find-defs-defun-defvar-el
   FAILED  xref-elisp-test-find-defs-defun-el
   FAILED  xref-elisp-test-find-defs-defvar-el
   FAILED  xref-elisp-test-find-defs-feature-el

But all five of the failing tests in help-fns-tests.el still fail.

However, concerning the latter, it seems that the tests expect
help-fns-function-description-header to return the quoted basename of
the file, while in my environment it returns the absolute filename.
Adjusting the regexp used in the tests fixes this issue (see the first
patch below) and then all tests pass regardless of where I start the
batch run.

Concerning cl-generic-tests.el, using file-truename as in the second
patch below prevents the one failure, also regardless of where I start
the batch run.

As for the failing xref tests in elisp-mode-tests.el, by applying
file-truename to the result of calls to find-lisp-object-file-name in
elisp--xref-find-definitions (in elisp-mode.el) and
xref-mode-local-overload (in mode-local.el) I could reduce the 14
failures to the same five that fail when doing the batch run from the
real directory without following the symlink.  The same reduction but
not full elimination is achieved by removing the call to file-truename
in the value of the defvar emacs-test-dir in elisp-mode-tests.el.  So
far I haven't figured out how to prevent these five failures.


diff --git a/test/lisp/help-fns-tests.el b/test/lisp/help-fns-tests.el
index da2b49e6b8..7782a41b9f 100644
--- a/test/lisp/help-fns-tests.el
+++ b/test/lisp/help-fns-tests.el
@@ -56,28 +56,28 @@ help-fns-test-interactive-built-in
     (should (string-match regexp result))))
 
 (ert-deftest help-fns-test-lisp-macro ()
-  (let ((regexp "a Lisp macro in .subr\\.el")
+  (let ((regexp "a Lisp macro in .+subr\\.el")
         (result (help-fns-tests--describe-function 'when)))
     (should (string-match regexp result))))
 
 (ert-deftest help-fns-test-lisp-defun ()
-  (let ((regexp "a compiled Lisp function in .subr\\.el")
+  (let ((regexp "a compiled Lisp function in .+subr\\.el")
         (result (help-fns-tests--describe-function 'last)))
     (should (string-match regexp result))))
 
 (ert-deftest help-fns-test-lisp-defsubst ()
-  (let ((regexp "a compiled Lisp function in .subr\\.el")
+  (let ((regexp "a compiled Lisp function in .+subr\\.el")
         (result (help-fns-tests--describe-function 'posn-window)))
     (should (string-match regexp result))))
 
 (ert-deftest help-fns-test-alias-to-defun ()
-  (let ((regexp "an alias for .set-file-modes. in .subr\\.el")
+  (let ((regexp "an alias for .set-file-modes. in .+subr\\.el")
         (result (help-fns-tests--describe-function 'chmod)))
     (should (string-match regexp result))))
 
 (ert-deftest help-fns-test-bug23887 ()
   "Test for https://debbugs.gnu.org/23887 ."
-  (let ((regexp "an alias for .re-search-forward. in .subr\\.el")
+  (let ((regexp "an alias for .re-search-forward. in .+subr\\.el")
         (result (help-fns-tests--describe-function 'search-forward-regexp)))
     (should (string-match regexp result))))
 

diff --git a/test/lisp/emacs-lisp/cl-generic-tests.el 
b/test/lisp/emacs-lisp/cl-generic-tests.el
index 5aa58782f3..9582907e51 100644
--- a/test/lisp/emacs-lisp/cl-generic-tests.el
+++ b/test/lisp/emacs-lisp/cl-generic-tests.el
@@ -240,7 +240,7 @@ cl-generic-tests--method-files--finds-methods
   (let ((retval (cl--generic-method-files 'cl-generic-tests--generic)))
     (should (equal (length retval) 2))
     (mapc (lambda (x)
-            (should (equal (car x) cl-generic-tests--this-file))
+            (should (equal (file-truename (car x)) 
cl-generic-tests--this-file))
             (should (equal (cadr x) 'cl-generic-tests--generic)))
           retval)
     (should-not (equal (nth 0 retval) (nth 1 retval)))))


In GNU Emacs 28.0.50 (build 16, x86_64-pc-linux-gnu, GTK+ Version 3.24.17, 
cairo version 1.17.3)
 of 2020-08-21 built on strobe-jhalfs
Repository revision: 3e10174fb65f4eb601b1921271bdcf10c933b879
Repository branch: master
Windowing system distributor 'The X.Org Foundation', version 11.0.12008000
System Description: Linux From Scratch SVN-20200401

Configured using:
 'configure 'CFLAGS=-Og -g3' PKG_CONFIG_PATH=/opt/qt5/lib/pkgconfig'

Configured features:
XPM JPEG TIFF GIF PNG RSVG CAIRO SOUND DBUS GSETTINGS GLIB NOTIFY
INOTIFY ACL GNUTLS LIBXML2 FREETYPE HARFBUZZ ZLIB TOOLKIT_SCROLL_BARS
GTK3 X11 XDBE XIM MODULES THREADS LIBSYSTEMD PDUMPER LCMS2