Re: python-mode: make sure output is not eaten

[Stefan Monnier]

> #> > Not, not really -- it looks to me like an attempt to fix some
> #> > prompt problems which didn't quite work. Or it may be an attempt to
> #> > have C-c C-s work for indented blocks, as Steven suggested.
> #> 
> #> I've installed a half-way solution which I believe keeps the old
> #> behavior when needed.

> Have you installed the complete patch? Because current version of the
> CVS code doesn't quite work for me... it hangs emacs whenever I try
> python-send-buffer (I need C-g to get it make it responsive again),
> which seems to be related to the "while" loop in python-send-receive
> never finishing due to the (local-variable-p 'python-preoutput-result)
> in python-preoutput-filter never being true...

Hmm... looks like the two pieces of code don't run in the same buffer maybe?
I've just installed a patch which may fix it.

> I still think 
>   (python-send-string "import emacs")
>   (python-send-receive "print '_emacs_out ()'")
> is cleaner than
>   (python-send-receive "import emacs; print '_emacs_out ()'")

Again, here's the issue: the process's output can be absolutely arbitrary.
If we see "\n>>> " somewhere in the process's output we have in general no
guarantee that it's actually a prompt.  Similarly if we see "_emacs_out " we
have in general no guarantee that it's coming from python.el or from
emacs.py.  So we should be careful in the process filter to be as selective
as possible in what we remove.

In interactive use, it's perfectly OK to do:

   >>> import emacs
   >>> print '_emacs_out ()'
   _emacs_out ()
   >>> 

but from python.el this basically turns into:

   >>> import emacs
   print '_emacs_out ()'
   >>> _emacs_out ()
   >>> 

because we don't wait for the prompt in between the two.  That's not
something you'd normally do interactively.  Instead if you didn't want to
wait for the prompt between the two, you'd do something like:

   >>> import emacs; print '_emacs_out ()'
   _emacs_out ()
   >>> 

which has the advantage that the _emacs_out () thingy appears at its normal
place so you can be more selective and risk misidentifying it in
fewer cases.

> I do not understand why "command" python-send-command shouldn't contain
> newline...

It's not a command, so it can only called from elisp code.  Check the
callers and you'll see it is never called with a newline.  So it's OK to
make such an assumption.

> It should work just fine with multiple lines, the only thing which might
> cause problems is indentation, and even that only if there is an indented
> block at the very end.

Sending a multiline thingy via python-send-string will look like absolute
crap in the resulting output, because all the intermediate prompts will get
concatenated rather than interleaved with the input.  So I see no point in
trying to support this in a case such as `python-send-command' where we
don't use this flexibility anyway (and where we need to analyze the output,
so the more control over it the better).

> If you are thinking about conditionally adding one or two newlines after
> python-send-string, than the correct condition should be "add double
> newline if and only if the last line is indented", IMHO.

Sure, go ahead.  I just tried a simple heuristic to preserve the previous
"\n\n" thingy without imposing it in the case where I know it's unneeded and
even annoying.

> I do not understand the purpose of python-preoutput-result variable. It
> is used in python-send-receive and in python-preoutput-filter, but those
> uses seem unrelated...

They're very much related: the accept-process-output call in
python-send-receive will let process filters run, so it's basically an
indirect call to python-preoutput-filter.

> if I read the code correctly, python-send-receive
> will call (make-local-variable 'python-preoutput-filter), but it will
> also always call (kill-local-variable 'python-preoutput-filter)... what
> is the point of it? And when could testing (local-variable-p
> 'python-preoutput-result) in python-preoutput-filter make sense?

Yes, it checks that the filter is being run from within the
accept-process-output of python-send-receive.

> Also, python-mode-running doesn't seem to be used anywhere.

It's used right where it's used ;-)

  (unless (boundp 'python-mode-running) ; kill the recursion from jython-mode
    (let ((python-mode-running t))
      (python-maybe-jython))))

i.e. it's used to prevent python-maybe-jython -> jython-mode ->
python-mode -> python-maybe-jython -> jython-mode -> python-mode ->
python-maybe-jython -> ...

> #> > Not really, at least not in my testing... the problems I am seeing
> #> > come from the fact that when eldoc sends commands, Python responds
> #> > with line that looks like "_emacs_out reload(module)\n>>> " and the
> #> > comint filter needs to remove the trailing prompt as well.
> #> 
> #> I've added python-preoutput-skip-next-prompt for that purpose.  This way
> #> such prompts should get removed unambiguously.

> I do not think this is the right way to solve the problem. IMHO, the
> variable should be called python-inhibit-output and should prevent
> *anything* from being printed in Inferior Python buffer -- thus,
> functions like eldoc could bind it to 't and we would not need to worry
> about them at all.

> After all, it doesn't matter how the output of python-eldoc-function
> looks like, it should never appear in the comint buffer. And it would
> let us simplify the common case of *user* wanting to run Python code.

Sounds good.  As long as you can be sure that there can be no other output
at the same time (i.e. the python process is idle, waiting at the prompt
when you send the eargs command).  Currently, we do not check that python is
idle, so we have no guarantee that we can just discard all the output
between after we send the `eargs' (ou modpath, or complete, ...) until after
we see the _emacs_out.

> #> > However, if you change the
> #> >    (string-match "\\`_emacs_out \\(.*\\)\n\\'" line)
> #> > to
> #> >    (string-match "\\`\\(?:>>> \\)?_emacs_out \\(.*\\)\n\\'" line)
> #> 
> #> > then you do not need to do the trick of augmenting commands with ";
> #> > print" statement: things work just fine if you do python-send-string
> #> > followed by python-send-receive.
> #> 
> #> Silently interpreting/removing output from the process is always risky,
> #> since the output may appear for other reasons than the ones we expected.
> #> So I'd rather be extra careful to only recognize _emacs_out when I'm sure
> #> it's really the internal thingy.

> I do not quite understand... You never *know* that it is the internal
> thingy anyway (try typing "print '_emacs_out'" in the Inferior Python
> buffer)...

Thanks to the local-variable-p check, the filter should now correctly let it
go through, except for unusual cases.


        Stefan