bug#71017: [PATCH] Flow single-paragraph messages

[Sandra Snan]

This fixes two bugs when sending RFC 2646–formatted email.

First, the old code didn't refill or encode the last paragraph at all
unless there was at least one hard newline EOF. This was a bee to track
down because there were two separate issues at play. One was a a bug in
flow-fill.el where every paragraph except the last paragraph was
reflowed, but the last paragraph would stay hardwrapped. Manually
placing a hard newline at the end of the file was a workaround but I
don't always remember to do that. I managed to fix that bug a few months
ago.

Second, the old code borked up code indented with tabs and spaces
(iff that code had overly long lines), such as Lisp code. It could
sometimes insert extra whitespace in the middle of such long lines.
I fixed that bug shortly after the first one.

But for months dogfooding those two changes, sometimes a hardwrapped
email would still be sent. I finally managed to debug and figure it out
and it took all day. Turns out mml just plain didn't call the
fill-flowed-encode function if the message doesn't have any hard
newlines (newlines with the hard text property). Well, of course a
single-paragraph email isn't gonna have any hard newlines! But it still
needs reflowing!

So I've now changed that and updated the documentation to match those
news semantics. I went all the way, but a possible compromise might be
to not-flow a message that has \n\n but no hard text props, since that's
a sign that something is wrong.

Since the use-hard-newlines variable is buffer local and all this
reflowing is being done in a temp buffer, that variable is more than
useless so I've removed references to it.
---
 doc/misc/emacs-mime.texi |  7 ++---
 lisp/gnus/mml.el         | 29 ++++++++----------
 lisp/mail/flow-fill.el   | 65 +++++++++++++++++-----------------------
 3 files changed, 41 insertions(+), 60 deletions(-)

diff --git a/doc/misc/emacs-mime.texi b/doc/misc/emacs-mime.texi
index ef7ea61..7621a9a 100644
--- a/doc/misc/emacs-mime.texi
+++ b/doc/misc/emacs-mime.texi
@@ -1087,13 +1087,10 @@ terminated by soft newline characters are filled 
together and wrapped
 after the column decided by @code{fill-flowed-encode-column}.
 Quotation marks (matching @samp{^>* ?}) are respected.  The variable
 controls how the text will look in a client that does not support
-flowed text, the default is to wrap after 66 characters.  If hard
-newline characters are not present in the buffer, no flow encoding
-occurs.
+flowed text, the default is to wrap after 66 characters.
 
 You can customize the value of the @code{mml-enable-flowed} variable
-to enable or disable the flowed encoding usage when newline
-characters are present in the buffer.
+to enable or disable the flowed encoding usage.
 
 On decoding flowed text, lines with soft newline characters are filled
 together and wrapped after the column decided by
diff --git a/lisp/gnus/mml.el b/lisp/gnus/mml.el
index e3bc393..2db39dc 100644
--- a/lisp/gnus/mml.el
+++ b/lisp/gnus/mml.el
@@ -691,23 +691,18 @@ type detected."
                   (t
                    ;; Only perform format=flowed filling on text/plain
                    ;; parts where there either isn't a format parameter
-                   ;; in the mml tag or it says "flowed" and there
-                   ;; actually are hard newlines in the text.
-                   (let (use-hard-newlines)
-                     (when (and mml-enable-flowed
-                                (string= type "text/plain")
-                                (not (string= (cdr (assq 'sign cont)) "pgp"))
-                                (or (null (assq 'format cont))
-                                    (string= (cdr (assq 'format cont))
-                                             "flowed"))
-                                (setq use-hard-newlines
-                                      (text-property-any
-                                       (point-min) (point-max) 'hard 't)))
-                       (fill-flowed-encode)
-                       ;; Indicate that `mml-insert-mime-headers' should
-                       ;; insert a "; format=flowed" string unless the
-                       ;; user has already specified it.
-                       (setq flowed (null (assq 'format cont)))))
+                   ;; in the mml tag or it says "flowed".
+                   (when (and mml-enable-flowed
+                              (string= type "text/plain")
+                              (not (string= (cdr (assq 'sign cont)) "pgp"))
+                              (or (null (assq 'format cont))
+                                  (string= (cdr (assq 'format cont))
+                                           "flowed")))
+                     (fill-flowed-encode)
+                     ;; Indicate that `mml-insert-mime-headers' should
+                     ;; insert a "; format=flowed" string unless the
+                     ;; user has already specified it.
+                     (setq flowed (null (assq 'format cont))))
                    ;; Prefer `utf-8' for text/calendar parts.
                    (if (or charset
                            (not (string= type "text/calendar")))
diff --git a/lisp/mail/flow-fill.el b/lisp/mail/flow-fill.el
index 919490e..5c9ae21 100644
--- a/lisp/mail/flow-fill.el
+++ b/lisp/mail/flow-fill.el
@@ -73,50 +73,39 @@ RFC 2646 suggests 66 characters for readability."
 ;;;###autoload
 (defun fill-flowed-encode (&optional buffer)
   (with-current-buffer (or buffer (current-buffer))
-    ;; No point in doing this unless hard newlines is used.
-    (when use-hard-newlines
-      (let ((start (point-min)) end)
-       ;; Go through each paragraph, filling it and adding SPC
-       ;; as the last character on each line.
-       (while (setq end (text-property-any start (point-max) 'hard 't))
-         (save-restriction
-           (narrow-to-region start end)
-           (let ((fill-column (eval fill-flowed-encode-column t)))
-             (fill-flowed-fill-buffer))
-           (goto-char (point-min))
-           (while (re-search-forward "\n" nil t)
-             (replace-match " \n" t t))
-           (goto-char (setq start (1+ (point-max)))))))
-      t)))
-
-(defun fill-flowed-fill-buffer ()
-  (let ((prefix nil)
-       (prev-prefix nil)
-       (start (point-min)))
-    (goto-char (point-min))
-    (while (not (eobp))
-      (setq prefix (and (looking-at "[> ]+")
-                       (match-string 0)))
-      (if (equal prefix prev-prefix)
-         (forward-line 1)
+    (let ((fill-column (eval fill-flowed-encode-column t))
+         (start (point-min))
+         end)
+      ;; Go through each paragraph, filling it and adding SPC
+      ;; as the last character on each line.
+      (while (and (< start (point-max))
+                 (setq end (or (text-property-any start (point-max) 'hard 't)
+                               (point-max))))
        (save-restriction
-         (narrow-to-region start (point))
-         (let ((fill-prefix prev-prefix))
-           (fill-region (point-min) (point-max) t 'nosqueeze 'to-eop))
-         (goto-char (point-max)))
-       (setq prev-prefix prefix
-             start (point))))
-    (save-restriction
-      (narrow-to-region start (point))
-      (let ((fill-prefix prev-prefix))
-       (fill-region (point-min) (point-max) t 'nosqueeze 'to-eop)))))
+         (narrow-to-region start end)
+         (let ((prefix
+                (concat "\n"
+                        (or (and (looking-at ">[> ]*")
+                                 (match-string 0)) ""))))
+           (goto-char start)
+           (while (search-forward prefix nil t)
+             (replace-match " " t t))
+           (goto-char start)
+           (while (< (+ (point) fill-column) (point-max))
+             (let ((start (point)))
+               (forward-char fill-column)
+               (when (search-backward " " start t)
+                 (forward-char)
+                 (insert prefix)))))
+         (setq start (1+ (point-max))))))
+    t))
 
 ;;;###autoload
 (defun fill-flowed (&optional buffer delete-space)
   "Apply RFC2646 decoding to BUFFER.
 If BUFFER is nil, default to the current buffer.
 
-If DELETE-SPACE, delete RFC2646 spaces padding at the end of
+If DELETE-SPACE, delete RFC3676 spaces padding at the end of
 lines."
   (with-current-buffer (or buffer (current-buffer))
     (let ((fill-column  (eval fill-flowed-display-column t)))
@@ -154,7 +143,7 @@ lines."
           ;; Delete the newline.
           (when (eq (following-char) ?\s)
             (delete-char 1))
-          ;; Hack: Don't do the flowing on the signature line.
+          ;; As per RFC3767: Don't do the flowing on the signature line.
           (when (and (not (looking-at "-- $"))
                      (eq (char-before (line-end-position)) ?\s))
             (while (and (not (eobp))
-- 
2.39.2