bug#69775: [PATCH] Use regexp-opt in dired-omit-regexp

[sbaugh]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: sbaugh@catern.com
>> Date: Sat, 16 Mar 2024 17:15:52 +0000 (UTC)
>> Cc: Spencer Baugh <sbaugh@janestreet.com>, 69775@debbugs.gnu.org
>> 
>> $ ./src/emacs -Q --batch -l ../emacs-29/bench-omit.elc
>> files 1, ntimes 100: new 0.008839979999999999 old 0.018162129 new-uncached 
>> 0.031399762
>> files 10, ntimes 100: new 0.012037615 old 0.040232355000000004 new-uncached 
>> 0.037990543
>> files 100, ntimes 100: new 0.07368538100000001 old 0.314905271 new-uncached 
>> 0.10006527300000001
>> files 1000, ntimes 100: new 0.669103498 old 3.076339984 new-uncached 
>> 0.693134644
>> files 10000, ntimes 100: new 6.336211434 old 30.926320486 new-uncached 
>> 6.442762152999999
>> 
>> So the performance improvement is quite substantial for large
>> directories.
>> 
>> new-uncached is the performance if dired-omit-extensions changes on each
>> call of dired-omit-regexp.  For a directory of 1 file, the overhead of
>> recomputing regexp-opt every time makes the performance perhaps 2x-3x
>> worse, but around 10 files the performance improvement from regexp-opt
>> exceeds the overhead, and above that the uncached version still
>> outperforms the old version substantially.
>
> SGTM, thanks.
>
>> Certainly, updated patch attached.
>> 
>> +@defvar dired-omit-size-limit
>> +If non-@code{nil}, omitting will be skipped if the directory listing
>> +exceeds this size in bytes.
>
> I'd rephrase
>
>    If non-@code{nil}, @code{dired-omit-mode} will be effectively
>    disabled in directories whose listing has size (in bytes) larger
>    than the value of this option.
>
>> +*** The default value of 'dired-omit-size-limit' has increased.
>                                                     ^^^
> "was" is better there.
>
>> +(defcustom dired-omit-size-limit 300000
>> +  "Maximum buffer size for `dired-omit-mode'.
>> +
>> +Omitting will be skipped if the directory listing exceeds this size in
>                     ^^^^^^^
> "disabled"

Adjusted all these, here's the new patch.

>From 7ae71f994814f3eb7bad3387608e8c0cbe0d2d68 Mon Sep 17 00:00:00 2001
From: Spencer Baugh <sbaugh@catern.com>
Date: Sat, 16 Mar 2024 17:11:24 +0000
Subject: [PATCH] Use regexp-opt in dired-omit-regexp

In my benchmarking, for large dired buffers, using regexp-opt provides
around a 3x speedup in omitting.

regexp-opt takes around 5 milliseconds, so to avoid slowing down
omitting in small dired buffers we cache the return value.

Since omitting is now 3x faster, increase dired-omit-size-limit by 3x.
Also, document dired-omit-size-limit better.

* doc/misc/dired-x.texi (Omitting Variables): Document
dired-omit-size-limit.
* etc/NEWS: Announce increase of dired-omit-size-limit.
* lisp/dired-x.el (dired-omit--extension-regexp-cache): Add.
(dired-omit-regexp): Use regexp-opt. (bug#69775)
(dired-omit-size-limit): Increase and improve docs.
---
 doc/misc/dired-x.texi |  9 +++++++++
 etc/NEWS              |  6 ++++++
 lisp/dired-x.el       | 26 ++++++++++++++++++++------
 3 files changed, 35 insertions(+), 6 deletions(-)

diff --git a/doc/misc/dired-x.texi b/doc/misc/dired-x.texi
index 4cad016a0f6..726b6653d0d 100644
--- a/doc/misc/dired-x.texi
+++ b/doc/misc/dired-x.texi
@@ -346,6 +346,15 @@ Omitting Variables
 match the file name relative to the buffer's top-level directory.
 @end defvar
 
+@defvar dired-omit-size-limit
+If non-@code{nil}, @code{dired-omit-mode} will be effectively disabled
+in directories whose listing has size (in bytes) larger than the value
+of this option.  Since omitting can be slow for very large directories,
+this avoids having to wait before seeing the directory.  This variable
+is ignored when @code{dired-omit-mode} is called interactively, such as
+by @code{C-x M-o}, so you can still enable omitting in the directory
+after the initial display.
+
 @cindex omitting additional files
 @defvar dired-omit-marker-char
 Temporary marker used by Dired to implement omitting.  Should never be used
diff --git a/etc/NEWS b/etc/NEWS
index 50f0ee4a1aa..9324d9f1dcf 100644
--- a/etc/NEWS
+++ b/etc/NEWS
@@ -692,6 +692,12 @@ marked or clicked on files according to the OS 
conventions.  For
 example, on systems supporting XDG, this runs 'xdg-open' on the
 files.
 
+*** The default value of 'dired-omit-size-limit' was increased.
+After performance improvements to omitting in large directories, the new
+default value is 300k, up from 100k.  This means 'dired-omit-mode' will
+omit files in directories whose directory listing is up to 300 kilobytes
+in size.
+
 +++
 *** 'dired-listing-switches' handles connection-local values if exist.
 This allows to customize different switches for different remote machines.
diff --git a/lisp/dired-x.el b/lisp/dired-x.el
index 62fdd916e69..753d3054d2f 100644
--- a/lisp/dired-x.el
+++ b/lisp/dired-x.el
@@ -77,12 +77,17 @@ dired-vm-read-only-folders
                 (other :tag "non-writable only" if-file-read-only))
   :group 'dired-x)
 
-(defcustom dired-omit-size-limit 100000
-  "Maximum size for the \"omitting\" feature.
+(defcustom dired-omit-size-limit 300000
+  "Maximum buffer size for `dired-omit-mode'.
+
+Omitting will be disabled if the directory listing exceeds this size in
+bytes.  This variable is ignored when `dired-omit-mode' is called
+interactively.
+
 If nil, there is no maximum size."
   :type '(choice (const :tag "no maximum" nil) integer)
   :group 'dired-x
-  :version "29.1")
+  :version "30.1")
 
 (defcustom dired-omit-case-fold 'filesystem
   "Determine whether \"omitting\" patterns are case-sensitive.
@@ -506,14 +511,23 @@ dired-omit-expunge
                                       (re-search-forward dired-re-mark nil 
t))))
         count)))
 
+(defvar dired-omit--extension-regexp-cache
+  nil
+  "A cache of `regexp-opt' applied to `dired-omit-extensions'.
+
+This is a cons whose car is a list of strings and whose cdr is a
+regexp produced by `regexp-opt'.")
+
 (defun dired-omit-regexp ()
+  (unless (equal dired-omit-extensions (car 
dired-omit--extension-regexp-cache))
+    (setq dired-omit--extension-regexp-cache
+          (cons dired-omit-extensions (regexp-opt dired-omit-extensions))))
   (concat (if dired-omit-files (concat "\\(" dired-omit-files "\\)") "")
           (if (and dired-omit-files dired-omit-extensions) "\\|" "")
           (if dired-omit-extensions
               (concat ".";; a non-extension part should exist
-                      "\\("
-                      (mapconcat 'regexp-quote dired-omit-extensions "\\|")
-                      "\\)$")
+                      (cdr dired-omit--extension-regexp-cache)
+                      "$")
             "")))
 
 ;; Returns t if any work was done, nil otherwise.
-- 
2.41.0