bug#71440: Python Inferior Mode Can’t Recognize My Prompt

[kobarity]

Eli Zaretskii wrote:
> 
> > Date: Wed, 12 Jun 2024 01:24:39 +0900
> > From: kobarity <kobarity@gmail.com>
> > Cc: 71440@debbugs.gnu.org
> > 
> > 
> > > Eli Zaretskii wrote:
> > > > > > I wonder how PS1 and PS2 are at all relevant when using 
> > > > > > python-shell-send-buffer?  That function sends the buffer
> > > > > > text to Python, so where do PS1 and PS2 come into play,
> > > > > > and why does Python say "__PYTHON_EL_eval is not defined" 
> > > > > > just because you have PS1 and PS2 customized?
> > 
> > PS1 and PS2 are set to `comint-prompt-regexp', and used to identify
> > execution completion.  The prompts must also be identified to
> > determine if the command can be sent.
> > 
> > Python REPL cannot accept the Python code as is.  For example, try
> > pasting the following code to the REPL:
> > 
> > if True:
> >     print("True")
> > print("Hi")
> > 
> > You should see the message "SyntaxError: invalid syntax".  This is
> > because the REPL requires an empty line to recognize the completion of
> > a block.  For such reasons, `python-shell-send-*' sends a string as
> > the argument to __PYTHON_EL_eval instead of sending the code as is.
> > 
> > __PYTHON_EL_eval is defined during the initialization of inferior
> > Python, but if the prompt is not recognized, its definition cannot be
> > done either.
> > 
> > The prompts are recognized by `python-shell-prompt-detect'.  A small
> > Python code sends PS1, etc. to Emacs as an array of JSON strings.
> > However, the JSON strings are hand-crafted for compatibility, as noted
> > in the comments below.
> > 
> >                     ;; JSON is built manually for compatibility
> > 
> > The json package was added in Python 2.6, so I assume it is to support
> > Python 2.5 and earlier.  This is fine for prompts consisting only of
> > ordinary characters, but will not result in a correct JSON string if
> > it contains escape sequences.
> > 
> > The attached improved patch uses the json package if available, so it
> > can handle escape sequences; without the json package, it works as
> > before.  It means that prompts containing escape sequences can be
> > recognized in Python 2.6 or later.  I have also added an ERT to check
> > this.
> 
> Thanks, but what do you mean by "json package" here?  Emacs nowadays
> has JSON capabilities built in, so no optional features should be
> required.  I did I misunderstand what you say above?

I meant the json package of the Python standard library.  A small
Python code is embedded in python.el.  I changed this Python code to
use the json package of the Python standard library if available.

It may be easier to see the actual code:

      (let* ((code (concat
                    "import sys\n"
                    "ps = [getattr(sys, 'ps%s' % i, '') for i in range(1,4)]\n"
                    "try:\n"
                    "    import json\n"
                    "    ps_json = '\\n' + json.dumps(ps)\n"
                    "except ImportError:\n"
                    ;; JSON is built manually for compatibility
                    "    ps_json = '\\n[\"%s\", \"%s\", \"%s\"]\\n' % 
tuple(ps)\n"
                    "\n"
                    "print (ps_json)\n"
                    "sys.exit(0)\n"))