bug#68559: [PATCH] Improve Python shell completion

[Liu Hui]

kobarity <kobarity@gmail.com> writes:

> I'm experiencing strange behavior regarding completion of import
> statement in a block in Python buffer.  If I try to type the following
> lines and then try to complete it, it will fail.
>
> #+begin_src python
> try:
>     im
> #+end_src

The problem can be reproduced by running python with jedi completer in
a terminal. The reason is that readline completer can only see the
text in the current line, i.e. "    im"; in this case, jedi does not
complete the text to "    import".

Similarly, the jedi completer cannot complete "except" after ex:

try:
    pass
ex|

> However, when I try to complete at the beginning of the second line:
>
> #+begin_src python
> try:
>
> #+end_src
>
>
> "import" keyword also appears as a candidate.  If I cancel the
> candidates and type "im" and try to complete it, it will succeed.

It is because jedi produces completions including "    import" for
blank string "    ". Due to the same prefix between "    " and
"    im", completion cache is reused for the latter. Then "    import"
can be completed.

It is more a limitation of readline completer than a problem with
jedi, as we cannot provide proper completion context for jedi. We may
define a custom completer to combine jedi and rlcompleter, e.g.

(setq python-shell-readline-completer "
def __PYTHON_EL_setup_readline_completer():
    import readline, rlcompleter
    import re, sys, os, __main__
    from jedi import Interpreter

    class MyJediRL:
        def __init__(self):
            self.rlcompleter = rlcompleter.Completer()
            self.rldelim = readline.get_completer_delims()

        def complete(self, text, state):
            if state == 0:
                sys.path.insert(0, os.getcwd())
                try:
                    interpreter = Interpreter(text, [__main__.__dict__])
                    completions = interpreter.complete(fuzzy=False)
                    self.matches = [
                        text[:len(text) - c._like_name_length] + 
c.name_with_symbols
                        for c in completions
                    ]

                    # try rlcompleter
                    sub = re.split('[' + re.escape(self.rldelim) + ']', 
text)[-1]
                    i = 0
                    while True:
                        completion = self.rlcompleter.complete(sub, i)
                        if not completion:
                            break
                        i += 1
                        completion = text[:len(text)-len(sub)] + 
completion.rstrip(' ()')
                        if completion not in self.matches:
                            self.matches.append(completion)
                except:
                    raise
                finally:
                    sys.path.pop(0)
            try:
                return self.matches[state]
            except IndexError:
                return None

    readline.set_completer(MyJediRL().complete)
    readline.set_completer_delims('')")

> Another thing I noticed is the multi-line import statement.  If the
> import statement is one-line, each items (IGNORECASE and MULTILINE in
> the example below) can be completed.
>
> #+begin_src python
> from re import IGNORECASE, MULTILINE
> #+end_src
>
>
> However, they cannot be completed if the import statement spans
> multi-line.
>
> #+begin_src python
> from re import (
>     IGN
> #+end_src
>
> This happens in both Python buffer and Python Shell buffer.  Perhaps
> this is a limitation of Jedi completer?

Yes. Because readline completer cannot see cross-line context, I added
the function "python-shell-completion-extra-context" in previous patch
to address the case of multi-line function call. I have updated the
attached patch to handle multi-line import statement.

The change to python-tests.el has been incorporated in the patch.
Thanks!

0001-Improve-Python-shell-completion-bug-68559.patch
Description: Text Data