Re: Org 9.6-pre and Bash sessions

[Ihor Radchenko]

Ihor Radchenko <yantar92@posteo.net> writes:

> Rudolf Adamkovič <salutis@me.com> writes:
>
>> :
>> : > > hello
>>
>> on the subsequent runs.
>>
>> Better than the new version but still wrong. :)
>
> And this is what drove me crazy during debugging. I do not understand
> the logic of all that ob-comint code.
>
> I have identified that the hang happens because Org does not change PS2
> prompt. Just PS1. But fixing this would yield to
>
> : org_babel_sh_prompt> org_babel_sh_prompt> hello
>
> Then, I tried to see how the original code works. And it does not
> :facepalm:
>
> I asked emacs-devel
> https://yhetil.org/emacs-devel/87y1tgqhmc.fsf@localhost/T/#u

See the attached tentative patch.
I'd appreciate some testing. Hopefully, I did not break anything.
Comint is tricky.

>From 4c6eead351cbdce1b9210a738c65b3a139d1cc0c Mon Sep 17 00:00:00 2001
Message-Id: 
<4c6eead351cbdce1b9210a738c65b3a139d1cc0c.1666330028.git.yantar92@posteo.net>
From: Ihor Radchenko <yantar92@posteo.net>
Date: Fri, 21 Oct 2022 13:21:57 +0800
Subject: [PATCH] ob-shell: Fix multi-line scripts in sessions
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

* lisp/ob-comint.el (org-babel-comint-with-output): Cleanup empty
output.  Such output is spitted unnecessarily for multi-line scripts.
* lisp/ob-shell.el (org-babel-shell-set-prompt-commands): Disable
PS2 and equivalent prompts.  Make sure that PROMPT_COMMAND does not
interfere with PS1 setting in POSIX shells.
(org-babel-sh-evaluate): Do not send input line-by-line.  Instead, let
`org-babel-coming-with-output' handle waiting for the output as well
as recording it.  Update to the new `org-babel-coming-with-output'
behavior of cleaning empty outputs.
* testing/lisp/test-ob-shell.el (test-ob-shell/session): Add test.

Reported-by: Rudolf Adamkovič <salutis@me.com>
Link: m2r0zboix1.fsf@me.com/">https://list.orgmode.org/orgmode/m2r0zboix1.fsf@me.com/
---
 lisp/ob-comint.el             |  2 +-
 lisp/ob-shell.el              | 29 ++++++++++++++---------------
 testing/lisp/test-ob-shell.el |  6 +++++-
 3 files changed, 20 insertions(+), 17 deletions(-)

diff --git a/lisp/ob-comint.el b/lisp/ob-comint.el
index 36a55d36c..ec7d3f6c9 100644
--- a/lisp/ob-comint.el
+++ b/lisp/ob-comint.el
@@ -109,7 +109,7 @@ (defmacro org-babel-comint-with-output (meta &rest body)
                      "\n" "[\r\n]+" (regexp-quote (or ,full-body "")))
                     string-buffer))
           (setq string-buffer (substring string-buffer (match-end 0))))
-        (split-string string-buffer comint-prompt-regexp)))))
+         (delete "" (split-string string-buffer comint-prompt-regexp))))))
 
 (defun org-babel-comint-input-command (buffer cmd)
   "Pass CMD to BUFFER.
diff --git a/lisp/ob-shell.el b/lisp/ob-shell.el
index 4d579ae87..d38d2d335 100644
--- a/lisp/ob-shell.el
+++ b/lisp/ob-shell.el
@@ -47,10 +47,15 @@ (defvar org-babel-default-header-args:shell '())
 (defvar org-babel-shell-names)
 
 (defconst org-babel-shell-set-prompt-commands
-  '(("fish" . "function fish_prompt\n\techo \"%s\"\nend")
-    ("csh" . "set prompt=\"%s\"")
+  '(;; Fish has no PS2 equivalent.
+    ("fish" . "function fish_prompt\n\techo \"%s\"\nend")
+    ;; prompt2 is like PS2 in POSIX shells.
+    ("csh" . "set prompt=\"%s\"\nset prompt2=\"\"")
+    ;; PowerShell, similar to fish, does not have PS2 equivalent.
     ("posh" . "function prompt { \"%s\" }")
-    (t . "PS1=\"%s\""))
+    ;; PROMPT_COMMAND can override PS1 settings.  Disable it.
+    ;; Disable PS2 to avoid garbage in multi-line inputs.
+    (t . "PROMPT_COMMAND=;PS1=\"%s\";PS2="))
   "Alist assigning shells with their prompt setting command.
 
 Each element of the alist associates a shell type from
@@ -299,20 +304,14 @@ (defun org-babel-sh-evaluate (session body &optional 
params stdin cmdline)
             #'org-babel-sh-strip-weird-long-prompt
             (mapcar
              #'org-trim
-             (butlast
+             (butlast ; Remove eoe indicator
               (org-babel-comint-with-output
                   (session org-babel-sh-eoe-output t body)
-                (dolist (line (append (split-string (org-trim body) "\n")
-                                      (list org-babel-sh-eoe-indicator)))
-                  (insert line)
-                  (comint-send-input nil t)
-                  (while (save-excursion
-                           (goto-char comint-last-input-end)
-                           (not (re-search-forward
-                                 comint-prompt-regexp nil t)))
-                    (accept-process-output
-                     (get-buffer-process (current-buffer))))))
-              2))
+                 (insert (org-trim body) "\n"
+                         org-babel-sh-eoe-indicator)
+                (comint-send-input nil t))
+               ;; Remove `org-babel-sh-eoe-indicator' output line.
+              1))
             "\n"))
           ;; External shell script, with or without a predefined
           ;; shebang.
diff --git a/testing/lisp/test-ob-shell.el b/testing/lisp/test-ob-shell.el
index a0d5a8d22..05c369174 100644
--- a/testing/lisp/test-ob-shell.el
+++ b/testing/lisp/test-ob-shell.el
@@ -46,7 +46,11 @@ (ert-deftest test-ob-shell/session ()
 ob-comint.el, which was not previously tested."
   (let ((res (org-babel-execute:sh "echo 1; echo 2" '((:session . "yes")))))
     (should res)
-    (should (listp res))))
+    (should (listp res)))
+  ;; Test multi-line input.
+  (let ((res (org-babel-execute:sh "if true; then\necho \"yes\"\nfi" 
'((:session . "yes")))))
+    (should res)
+    (should (string= "yes" res))))
 
 ; A list of tests using the samples in ob-shell-test.org
 (ert-deftest ob-shell/generic-uses-no-arrays ()
-- 
2.35.1


-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>