Hi,
There are some problems in filling Python's strings, which are caused
by the same portion of the code.
Here are the steps to reproduce the first problem:
1. emacs -Q
2. C-x C-f somefile1.py
3. Enter the following program:
#+begin_src python
s1 = 'abc'
s2 = 'def'
#+end_src
4. Place point within the first string (abc).
6. M-q
The lines will be merged into a single line.
#+begin_src python
s1 = 'abc' s2 = 'def'
#+end_src
This problem occurs in Emacs 27 and later.
Second problem can be reproduced with the following steps:
1. emacs -Q
2. C-x C-f somefile2.py
3. Enter the following program:
#+begin_src python
s = '''abcdefghijklmnopqrstuvwxyz abcdefghijklmnopqrstuvwxyz
abcdefghijklmnopqrstuvwxyz'''
#+end_src
4. Place point at the first or third quote of the starting triple quotes.
5. M-q
The line is not filled. However, if the point is placed at the second
quote or within the string, the line will be filled as follows:
#+begin_src python
s = '''abcdefghijklmnopqrstuvwxyz abcdefghijklmnopqrstuvwxyz
abcdefghijklmnopqrstuvwxyz'''
#+end_src
When the point is placed at the ending triple quotes, the line will be
filled if the point is at the first or third quote.
This problem occurs in Emacs 28 and later.
Third problem can be reproduced with the following steps:
1. emacs -Q
2. C-x C-f somefile3.py
3. Enter the following program:
#+begin_src python
s = 'abcdefghijklmnopqrstuvwxyz abcdefghijklmnopqrstuvwxyz
abcdefghijklmnopqrstuvwxyz'
#+end_src
4. Place point within the string.
5. M-q
The line will be split into 2 lines.
#+begin_src python
s = 'abcdefghijklmnopqrstuvwxyz abcdefghijklmnopqrstuvwxyz
abcdefghijklmnopqrstuvwxyz'
#+end_src
This is not a correct Python program anymore because a single quoted
string can not span multiple lines unless the end of line is escaped
using a backslash.
The docstring of `python-fill-paragraph' says:
`fill-paragraph-function' handling multi-line strings and possibly comments.
Although it is possible to make a multi-line single quoted string by
escaping the end of line using backslash, triple quoted strings are
usually used for multi-line strings. So I think
`python-fill-paragraph' should not do anything with single quoted
strings.
The cause of the first problem is similar to Bug#58780. In
`python-fill-string', the string is searched for with the following
code:
(or (python-syntax-context 'string)
(and (equal (string-to-syntax "|")
(syntax-after (point)))
(point)))))
and the end of the string is searched for with the following code:
(or (re-search-forward (rx (syntax string-delimiter)) nil t)
(goto-char (point-max)))
These codes used to be appropriate, but it is not correct after the
following commit which is included in Emacs 27 and later:
commit 4b39b741f1949ebad1dfccc5032dfce521bedc2a
Author: Stefan Monnier <monnier@iro.umontreal.ca>
Date: Tue Apr 9 15:08:21 2019 -0400
python.el: don't syntax-propertize single/double quoted strings
* lisp/progmodes/python.el (python-syntax-propertize-function):
Only mark triple-quoted strings, let the normal syntax-table handle
the rest.
(python-syntax-stringify): Adjust accordingly.
From this commit, single quoted strings can not be identified by
string-delimiter and can only be identified by string-quote.
In the fix of Bug#58780, I added `forward-sexp' to search for the end
of single quoted string. However, considering the third problem, I
would prefer to have `python-fill-paragraph' and `python-fill-string'
search only for triple quoted strings in this case.
The second problem occurs after the following commit included in Emacs
28 and later:
commit 0646c6817139aa905a2f6079fdc82eb4be944de0
Author: João Távora <joaotavora@gmail.com>
Date: Sun Sep 19 11:42:20 2021 +0100
Make syntax-ppss more accurate for Python triple quotes (bug#49518)
By putting delimiter syntax on the last character of Python
triple-quoted strings, this makes syntax-ppss be more accurate.
This commit narrows the range of string-delimiter from outermost
quotes to innermost quotes, and results in `python-fill-paragraph' not
recognizing the string when point is at some quotes.
Attached is a patch to fix these problems. Main changes are:
1. single-quoted-string and triple-quoted-string are added as a TYPE
argument of `python-syntax-context'.
2. New helper function `python-info-triple-quoted-string-p' is added
to detect triple quoted strings including quotes.
3. `python-fill-paragraph' and `python-fill-string' use
`python-info-triple-quoted-string-p' to detect triple quoted
strings only.
I look forward to your comments.
--
In GNU Emacs 30.0.50 (build 3, x86_64-pc-linux-gnu, X toolkit, cairo
version 1.16.0, Xaw scroll bars) of 2023-03-10 built on ubuntu
Repository revision: 8ee205d232574e12921b052c7e93b7e16d6f1187
Repository branch: master
System Description: Ubuntu 22.04.2 LTS
Configured features:
ACL CAIRO FREETYPE GIF GLIB GMP GNUTLS GSETTINGS HARFBUZZ JPEG
LIBSELINUX LIBXML2 MODULES NOTIFY INOTIFY PDUMPER PNG SECCOMP SOUND
SQLITE3 THREADS TIFF TOOLKIT_SCROLL_BARS TREE_SITTER X11 XDBE XIM XPM
LUCID ZLIB
Important settings:
value of $LANG: en_US.UTF-8
locale-coding-system: utf-8-unix
Major mode: Python
Minor modes in effect:
tooltip-mode: t
global-eldoc-mode: t
eldoc-mode: t
show-paren-mode: t
electric-indent-mode: t
mouse-wheel-mode: t
tool-bar-mode: t
menu-bar-mode: t
file-name-shadow-mode: t
global-font-lock-mode: t
font-lock-mode: t
blink-cursor-mode: t
line-number-mode: t
transient-mark-mode: t
auto-composition-mode: t
auto-encryption-mode: t
auto-compression-mode: t
Load-path shadows:
None found.
Features:
(shadow sort mail-extr emacsbug message mailcap yank-media puny dired
dired-loaddefs rfc822 mml mml-sec password-cache epa derived epg rfc6068
epg-config gnus-util text-property-search time-date mm-decode mm-bodies
mm-encode mail-parse rfc2231 mailabbrev gmm-utils mailheader sendmail
rfc2047 rfc2045 ietf-drums mm-util mail-prsvr mail-utils python json
subr-x map rx project pcase treesit cl-seq comint ansi-osc ring
cl-loaddefs cl-lib ansi-color term/screen term/xterm xterm byte-opt gv
bytecomp byte-compile rmc iso-transl tooltip cconv eldoc paren electric
uniquify ediff-hook vc-hooks lisp-float-type elisp-mode mwheel
term/x-win x-win term/common-win x-dnd tool-bar dnd fontset image
regexp-opt fringe tabulated-list replace newcomment text-mode lisp-mode
prog-mode register page tab-bar menu-bar rfn-eshadow isearch easymenu
timer select scroll-bar mouse jit-lock font-lock syntax font-core
term/tty-colors frame minibuffer nadvice seq simple cl-generic
indonesian philippine cham georgian utf-8-lang misc-lang vietnamese
tibetan thai tai-viet lao korean japanese eucjp-ms cp51932 hebrew greek
romanian slovak czech european ethiopic indian cyrillic chinese
composite emoji-zwj charscript charprop case-table epa-hook
jka-cmpr-hook help abbrev obarray oclosure cl-preloaded button loaddefs
theme-loaddefs faces cus-face macroexp files window text-properties
overlay sha1 md5 base64 format env code-pages mule custom widget keymap
hashtable-print-readable backquote threads inotify dynamic-setting
system-font-setting font-render-setting cairo x-toolkit x multi-tty
make-network-process emacs)
Memory information:
((conses 16 60054 7620)
(symbols 48 7362 0)
(strings 32 20143 1278)
(string-bytes 1 593570)
(vectors 16 11583)
(vector-slots 8 139242 9455)
(floats 8 30 11543)
(intervals 56 239 0)
(buffers 984 12))
0001-Fix-python-fill-paragraph-problems-on-filling-string.patch
Description: Binary data
--- End Message ---