bug#68559: [PATCH] Improve Python shell completion

[kobarity]

Eli Zaretskii wrote:
> > 0001-Remove-echoed-back-string-in-python-shell-completion.patch
> > extracts only the last line to exclude echoed back strings.
> > 
> > 0001-Set-tty-mode-to-raw-when-setting-up-Inferior-Python.patch sets
> > the Inferior Python tty to raw mode.  python-ffap-module-path-1 will
> > no longer need to be skipped on Mac.  If it is safe to set tty to raw
> > mode on all UNIX based systems, I prefer this method.
> 
> Will this work on MS-Windows as well?  If you are unsure, would you
> please tell me how to test whether this works on Windows, so I could
> collect the information for you?

It does not affect MS-Windows because it sets only when the tty
library can be imported.  Although the tty library exists in
MS-Windows, importing the tty library will result in an error because
there is no underlying termios library.

Mattias Engdegård wrote:
> 
> 16 feb. 2024 kl. 16.24 skrev kobarity <kobarity@gmail.com>:
> 
> > I made prototype patches for each method.  I don't use Mac so it would
> > be helpful if you could try these.
> 
> Nice, thank you! I can confirm that they both appear to work, at least in the 
> sense that the python-tests pass (except for the ones skipped intentionally), 
> and the python shell behaves reasonable.
> 
> Only the set-tty-mode patch eliminates echo in the interactive python shell; 
> in that sense it's preferable.

Thank you for testing my patches.

> Both produce the very annoying warning
> 
>   Warning (python): Your ‘python-shell-interpreter’ doesn’t seem to support 
> readline, yet ‘python-shell-completion-native-enable’ was t and "python3" is 
> not part of the ‘python-shell-completion-native-disabled-interpreters’ list.  
> Native completions have been disabled locally. Consider installing the python 
> package "readline".
> 
> which is not even correct since the standard Python does have a working 
> readline module, even if it uses libedit.

You are right. Maybe the package name should be fixed to "gnureadline"
as described in etc/PROBLEMS.

On Mac, it might be better to set the default value of
`python-shell-completion-native-enable' to nil.

> > 0001-Set-tty-mode-to-raw-when-setting-up-Inferior-Python.patch sets
> > the Inferior Python tty to raw mode.  python-ffap-module-path-1 will
> > no longer need to be skipped on Mac.
> 
> Right, that test runs and passes.
> 
> >  If it is safe to set tty to raw
> > mode on all UNIX based systems, I prefer this method.
> 
> Same here. I see no reason why it wouldn't be safe, either.

Thanks.

> > By the way, is it necessary to send
> > `python-shell-completion-setup-code' for every completion in
> > `python-shell-completion-get-completions'?  To me it seems sufficient
> > to send it once at initialization.
> 
> Indeed, it does seem a bit extravagant.

This would be one of the items for future improvement.

Liu Hui wrote:
> 
> kobarity <kobarity@gmail.com> writes:
> 
> > 0001-Set-tty-mode-to-raw-when-setting-up-Inferior-Python.patch sets
> > the Inferior Python tty to raw mode.  python-ffap-module-path-1 will
> > no longer need to be skipped on Mac.  If it is safe to set tty to raw
> > mode on all UNIX based systems, I prefer this method.
> 
> How about the following change, which only affects libedit-based
> readline? It may enable native completion on mac, but I cannot test
> it.
> 
> diff --git a/lisp/progmodes/python.el b/lisp/progmodes/python.el
> index b7e43f3fc68..f59bc19367b 100644
> --- a/lisp/progmodes/python.el
> +++ b/lisp/progmodes/python.el
> @@ -4286,6 +4286,9 @@ (defcustom python-shell-completion-setup-code
>              except:
>                  pass
>          else:
> +            if readline.__doc__ and 'libedit' in readline.__doc__:
> +                import tty
> +                tty.setraw(0)
>              # Try to reuse current completer.
>              completer = readline.get_completer()
>              if not completer:
> @@ -4471,8 +4474,8 @@ (defun python-shell-completion-native-setup ()
>                  instance.rlcomplete = new_completer
> 
>          if readline.__doc__ and 'libedit' in readline.__doc__:
> -            raise Exception('''libedit based readline is known not to work,
> -      see etc/PROBLEMS under \"In Inferior Python mode, input is 
> echoed\".''')
> +            import tty
> +            tty.setraw(0)
>              readline.parse_and_bind('bind ^I rl_complete')
>          else:
>              readline.parse_and_bind('tab: complete')

Disabling echo back may not be sufficient to enable native completions
on Mac. I have not tried raw mode, but have tried
readline.parse_and_bind('setty -echo') and
readline.parse_and_bind('edit on').  Native completions could be
enabled, but it was unstable.

I have no objection if Mac users check the above patch and if it is
OK.