Comparison of tools to search for related files

[Damien Cassou]

Hi,

I'm implementing jumprel to jump from a file to related files (e.g., its
tests, its CSS, its .h header file…). During discussion for bug#57564,
Eli asked me

      How will this be different from what we already have:

      • the find-file.el package
      • the new command 'find-sibling-file'

I didn't know about these 2 packages so thank you very much for telling
me about them. Because I just learned about them, my description and
comparison below might be incomplete.

In the following I will compare the packages according to my 2 use
cases:

1. In the Emacs core code base, I want to jump from an elisp file (e.g.,
   `lisp/calendar/parse-time.el') to its test file (e.g.,
   `test/lisp/calendar/parse-time-tests.el', note the parallel folder
   hierarchy) and back.

2. In my JavaScript frontend project, I want to jump from a component
   file (e.g., `foo/MyComponent.js') to its Less file (e.g.,
   `foo/MyComponent.less', same folder and different extension) or to
   its UI test file (e.g., `foo/MyComponent.spec.component.js', same
   folder, same extension but different suffix) or to its non-UI test
   file (e.g., `test/foo/myComponent-tests.js', different casing and
   parallel folder hierarchy).

In both use cases, it would be nice to facilitate the creation of
non-existing files: for example, if a buffer visits `MyComponent.js' and
there is no `MyComponent.less', it would be great if Emacs could let me
create it from a list of non-existing related files.

I derive the following required features from these 2 use cases:

Parallel folder hierarchy
      It should be possible to jump from a file in `foo/bar/' to a file
      in `XX/foo/bar/' and back ("XX/" usually equals "test/" or
      "tests/");
Choose candidate
      The user should be presented with a list of related file
      candidates to pick the one to jump to;
Case changing
      Some related files might have a different casing;
Creation
      The user should be presented with the related files that don't
      exist so they can be created automatically (very useful with
      parallel folder hierarchies). I don't consider that a must but a
      nice to have.


1 find-file.el
══════════════

  find-file.el provides `ff-find-other-file' to jump from a file to a
  related file. How a file relates to another is done through
  `ff-other-file-alist' where elements can have 2 different forms:

  ┌────
  │ (REGEXP (EXTENSION...))
  │ (REGEXP FUNCTION)
  └────

  The first form above associates a filename regexp to a list of related
  file extensions. For example, `("\\.c\\'" (".h"))' associates a C file
  to its header file (it seems that C and C++ projects where the main
  target for this package).

  I didn't manage to configure find-file.el for parallel folder
  hierarchies. find-file.el provides `ff-search-directories' but it's
  not completely clear to me if this variable would make it possible to
  implement the use cases using the first form above (aka without
  relying on a function). I have the impression that it wouldn't.

  I managed to configure find-file.el for the cases when files are in
  the same folder as in most of the second use case:

  ┌────
  │ (("\\.spec.component\\.js\\'" (".js" ".less"))
  │  ("\\.less\\'" (".js" ".spec.component.js"))
  │  ("\\.js\\'" (".less" ".spec.component.js")))
  └────

  Unfortunately, only the first file matching from the EXTENSION list is
  used and the user is never presented a choice of candidate. Looking at
  the third line above, this means that if I'm in `MyComponent.js' and
  `MyComponent.less' exists, there is no way to go to
  `MyComponent.spec.component.js'.

  To implement parallel folder hierarchy, it is possible to use the
  second as it allows for more flexibility. The Emacs core use case
  would be implemented with:

  ┌────
  │ (defun my/ff-other-file-for-emacs-core (filename)
  │   (save-match-data
  │     (if (string-match "test/lisp" filename)
  │         (let ((without-test-directory (replace-match "lisp" nil t 
filename)))
  │           (list (replace-regexp-in-string "-tests\\.el$" ".el" 
without-test-directory)))
  │       (let ((with-test-directory (string-replace "/lisp/" "/test/lisp/" 
filename)))
  │         (list (replace-regexp-in-string "\\.el$" "-tests.el" 
with-test-directory))))))
  └────

  I have the impression that relying on elisp for such use cases will
  limit the usage of the package. For example, I haven't found any place
  in Emacs core code base or documentation where such a function would
  be given to facilitate the life of Emacs core contributors.

  find-file.el has a `ff-file-created-hook' variable to create and
  populate a file if no file exists. While a hook variable allows the
  user to do whatever it wants, it also means the user must write even
  more elisp code to do so.

  find-file.el has a `ff-special-constructs' to match import/include
  lines and open the imported file when cursor is on such a line. This
  seems completely unrelated to the rest of the code in find-file.el
  though and other find-file.el mechanisms are ignored in this case. I'm
  not sure why this code is here.

  Summary of find-file.el:

  Parallel folder hierarchy
        Supported through the creation of functions only. This limits
        the usage of this feature to elisp developers;
  Choose candidate
        find-file.el opens the first existing file and never let the
        user choose;
  Case changing
        Supported through the creation of functions only.
  Creation
        A hook exists which means it is possible but is limited to elisp
        developers.

  Beyond features, I found the code hard to read with very long
  functions, a lot of state mutation and no unit test. The code is
  around 800-line long.


2 find-sibling-file
═══════════════════

  This "package" consists of an interactive function, a customizable
  variable and a helper function.

  The Emacs core use case can easily be implemented by configuring
  `find-sibling-rules':

  ┌────
  │ (("test/lisp/\\(.*\\)-tests\\.el$" "lisp/\\1.el")
  │  ("lisp/\\(.*\\)\\.el$" "test/lisp/\\1-tests.el"))
  └────

  This works great.

  Ignoring parallel folder hierarchy (implemented just like in the
  previous use case) and case changing, the second use case can be

  ┌────
  │ (("\\(.*\\)\\.spec.component\\.js\\'" "\\1.js" "\\1.less")
  │  ("\\(.*\\)\\.less\\'" "\\1.js" "\\1.spec.component.js")
  │  ("\\(.*\\)\\.js\\'" "\\1.spec.component.js" "\\1.less"))
  └────

  This works great but redundancy starts to be annoying. If you need to
  add `*.stories.js' files (as is the case for my JS project), you will
  start suffering.

  If several matching files exist, the user is prompted with a list of
  candidates to choose from.

  I haven't found a way to implement case changing and there is no
  creation mechanism either.

  When regexps are not enough, there is no fallback-to-function
  workaround as was the case with find-file.el. I don't doubt this can
  easily be implemented though.

  The package has a nice feature to let the user switch between the same
  file in two different projects (e.g., `emacs-src-27/lisp/abbrev.el'
  and `emacs-src-28/lisp/abbrev.el'). I don't need the feature but I can
  see how it can be useful.

  Summary of find-sibling-file:

  Parallel folder hierarchy
        Supported with (slightly redundant) regexps;
  Choose candidate
        Supported;
  Case changing
        Unsupported.
  Creation
        Unsupported.

  Beyond features, the code is really simple and only 89-line long. It
  has no unit test though (yet?).


3 jumprel
═════════

  This is the package I'm working on. It provides a command to jump to a
  related file among existing candidates. It features a Domain-Specific
  Language (DSL) to describe the relation between files. For Emacs core,
  it would look like this
  ┌────

  └────

  ┌────
  │ (filename :remove-suffix ".el" :add-suffix "-tests.el" :add-directory 
"test")
  └────

  This line represents a jumper and must be added to
  `jumprel-jumpers'. This can be done in a `.dir-locals.el' file for
  example. This line is in my opinion much easier to understand and
  modify than the alternatives of the other two packages. Please note
  that this line works to go from a file to its test file and back: this
  limits the redundancy noticed above.

  The JavaScript UI use case would be implemented with these jumpers:

  ┌────
  │ (filename :remove-suffix ".js" :add-suffix "-tests.js" :add-directory 
"tests" :case-transformer uncapitalize)
  │ (filename :remove-suffix ".js" :add-suffix ".spec.component.js")
  │ (filename :remove-suffix ".js" :add-suffix ".less")
  │ (filename :remove-suffix ".js" :add-suffix ".stories.js")
  └────

  Note that the first line shows an example of using case
  transformation.

  When several files exist, the user is presented with a list of
  candidates just like `find-sibling-file'.

  `find-file.el' natively supports C and C++-based projects (see
  `cc-other-file-alist'). A similar configuration can be achieved with
  jumprel through such a simple jumper:

  ┌────
  │ (filename :remove-suffix ".c" :add-suffix ".h")
  └────

  If the DSL doesn't support your use case, it is possible to fallback
  to implementing a function, just like with find-file.el (I would
  prefer a patch to improve the DSL when it makes sense though). Another
  possibility is to define another DSL. For example, contrary to the
  other two packages, jumprel doesn't have any support for regexp-based
  definitions of file relations. This can easily be implemented by
  defining a new DSL and leveraging `find-sibling-file-search':

  ┌────
  │ (cl-defmethod jumprel-apply ((jumper (head regexp)) place)
  │   "Apply JUMPER to PLACE and return a new place or nil."
  │   (find-sibling-file-search
  │    place
  │    (list (cdr jumper))))
  └────

  With this in place, users can now specify exactly the same patterns as
  they would in `find-sibling-rules'. For example, the jumper below lets
  the user switch between the same file in two different projects (e.g.,
  `emacs-src-27/lisp/abbrev.el' and `emacs-src-28/lisp/abbrev.el'):

  ┌────
  │ (regexp "emacs/[^/]+/\\(.*\\)\\'" "emacs/.*/\\1")
  └────

  Additionally, jumprel provides a simple mechanism to declare how to
  populate files. For Emacs core, the `.dir-locals.el' file could
  contain:

  ┌────
  │ (filename :remove-suffix ".el" :add-suffix "-tests.el" :add-directory 
"test" :filler auto-insert)
  └────

  The `:filler auto-insert' part indicates that `auto-insert' must be
  called when a test file is created. You can also specify a string
  instead of `auto-insert' to give a default content. The mechanism is
  extensible so I also implemented a way to populate a file based on a
  yasnippet snippet (in my `init.el' file):

  ┌────
  │ (cl-defmethod jumprel-maker-fill ((filler (head yasnippet)) 
&allow-other-keys &rest)
  │   (when-let* ((snippet (map-elt (cdr filler) :name)))
  │     (yas-expand-snippet (yas-lookup-snippet snippet major-mode))))
  └────

  Which means the user can now specify a yasnippet snippet in their
  `.dir-locals.el' file:

  ┌────
  │ (filename :remove-suffix ".js" :add-suffix ".spec.component.js" :filler 
(yasnippet :name "componentSpec"))
  └────


  Summary of jumprel:

  Parallel folder hierarchy
        Supported with a simple `:add-directory' directive.
  Choose candidate
        Supported.
  Case changing
        Supported with a simple `:case-transformer' directive.
  Creation
        Supported with a simple `:filler' directive.

  Beyond features, jumprel's code is 403-line long but isn't fully
  documented yet. There are 227 lines of unit tests.

-- 
Damien Cassou

"Success is the ability to go from one failure to another without
losing enthusiasm." --Winston Churchill