Re: [PATCH] Async evaluation in ob-shell

[Matt]

 ---- On Wed, 15 Feb 2023 10:08:47 -0500  Ihor Radchenko  wrote --- 
 > Matt matt@excalamus.com> writes:
 > 
 > > Checking the final result from the callback is trickier.   The following 
 > > works, but requires advice (which could potentially persist beyond the 
 > > test) and a delay (which slows testing overall and whose duration likely 
 > > depends on the hardware the test runs on).  Without the delay, I could not 
 > > get the callback to execute within the test.  It would execute when 
 > > running manually in an Org buffer, however.  I'm not sure why. 
 > 
 > Doesn't (while ... (sleep-for ...)) work?

I'm afraid I don't follow what you mean.

My biggest concern is the sleep-for itself.  Aside from the concerns above, the 
test doesn't work without it. 

For example, the check made by the advice looks for "1\n2\n".  I've changed the 
source block from "echo 1" to "echo 2"  to induce a failure (it will now 
receive "2\n2\n").  I re-evaluate the test and call M-: (ert 
"test-ob-shell/session-async-evaluation").  The test passes!  I can put a 
message or debug in the advice and see that it's never called.  However, if I 
uncomment the sleep-for, the advice runs, and the test fails as expected.

(ert-deftest test-ob-shell/session-async-evaluation ()
  (let ((session-name "test-ob-shell/session-async-evaluation")
        (kill-buffer-query-functions nil)
        result)
    ;; check callback return for correct results
    (advice-add
     'ob-shell-async-chunk-callback
     :filter-return
     (lambda (&rest r)
       (let ((result (car r)))
         (should (string= result "1\n2\n"))  ; this was previously an if 
statement
         result))
     `((name . ,session-name)))
    ;; always remove advice, regardless of test outcome
    (unwind-protect
        (org-test-with-temp-text
            (concat "#+begin_src sh :session " session-name " :async t
echo 2
echo 2<point>
#+end_src")
          ;; execute block; callback only executes when delay
          (setq result (org-trim (org-babel-execute-src-block)))
          (if (should
               (and 
                (string-match test-ob-shell/uuid-regex result)  ; block runs
                (not (sleep-for 0.1))                           ; callback 
doesn't fail
                ))                         
              (kill-buffer session-name)))
      (advice-remove 'ob-shell-async-chunk-callback session-name))))