Re: search-default-mode char-fold-to-regexp and Greek Extended block characters, Re: search-default-mode char-fold-to-regexp and Greek Extended block characters

[Robert Pluim]

>>>>> On Fri, 26 Jul 2019 00:35:51 +0300, Juri Linkov <address@hidden> said:

    Juri> If there are many such cases, then better to handle them 
automatically indeed
    Juri> (if this doesn't cause slowdown too much) instead of adding them one 
by one
    Juri> to the default values.  Does this handle ß as well?

There are 74, and I donʼt want to maintain such a list by hand :-).

ß is not a complex character, so is never looked at here. But if we
hoist the checking out of the loop over complex characters, we can
make that work as well (this supersedes my previous patch).

I have no idea of the performance impact of all this.

diff --git i/lisp/char-fold.el w/lisp/char-fold.el
index f379229e6c..bee485c8ed 100644
--- i/lisp/char-fold.el
+++ w/lisp/char-fold.el
@@ -49,6 +49,36 @@
                             (funcall func (car char) v table)))
                         table))
 
+      (map-char-table
+       (lambda (char category)
+         ;; If the uppercase version of a character is not a single
+         ;; character, we add a mapping from the first character of
+         ;; the uppercase version to the lowercase character. This
+         ;; handles eg ß => SS and ῗ => Ϊ́
+
+         (when (eq category 'Ll)
+           (let ((start char)
+                 (end char))
+             (when (consp char)
+               (setq start (car char)
+                     end (cdr char)))
+             (while (<= start end)
+               (let* ((str (char-to-string start))
+                      (upper (upcase str))
+                      (roundtrip (downcase upper)))
+                 (when (> (length roundtrip) 1)
+                   ;; Complex characters map to the decomposed version
+                   ;; + diacriticals, simple characters map to the
+                   ;; base char.  Also add the reverse mapping for
+                   ;; simple characters.
+                   (if (cdr (get-char-code-property start 'decomposition))
+                       (setq str roundtrip)
+                     (aset equiv start
+                           (cons roundtrip (aref equiv start))))
+                   (aset equiv (aref roundtrip 0)
+                         (cons str (aref equiv (aref roundtrip 0))))))
+               (setq start (+ 1 start))))))
+             (unicode-property-table-internal 'general-category))
       ;; Compile a list of all complex characters that each simple
       ;; character should match.
       ;; In summary this loop does 3 things:
diff --git i/test/lisp/char-fold-tests.el w/test/lisp/char-fold-tests.el
index e519435ef0..cf155d3cb5 100644
--- i/test/lisp/char-fold-tests.el
+++ w/test/lisp/char-fold-tests.el
@@ -154,9 +154,9 @@ char-fold--test-without-customization
             ("ι"
              "ί" ;; 1 level decomposition
              "ί" ;; 2 level decomposition
-             ;; FIXME:
-             ;; "ΐ" ;; 3 level decomposition
+             "ΐ" ;; 3 level decomposition
              )
+            ("ß" "ss")
             )))
     (dolist (strings matches)
       (apply 'char-fold--test-match-exactly strings))))
@@ -165,7 +165,6 @@ char-fold--test-with-customization
   :tags '(:expensive-test)
   (let* ((char-fold-include
           '(
-            (?ß "ss") ;; de
             (?o "ø")  ;; da no nb nn
             (?l "ł")  ;; pl
             ))
@@ -184,8 +183,7 @@ char-fold--test-with-customization
           '(
             ("e" "ℯ" "ḗ" "ë" "ë")
             ("е" "ё" "ё")
-            ("ι" "ί" "ί"
-             ;; FIXME: "ΐ"
+            ("ι" "ί" "ί" "ΐ"
              )
             ("ß" "ss")
             ("o" "ø")