emacs-orgmode
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Feedback on Emacs-Jupyter


From: David Dynerman
Subject: Re: Feedback on Emacs-Jupyter
Date: Wed, 5 Jan 2022 09:39:11 -0800

Dear Nathaniel,

Let me echo others’ comments by saying how much I appreciate your work on 
emacs-jupyter. I use the package daily in my work and it’s been really 
fantastic. Thank you very, very much for your huge efforts and time investment 
in creating and maintaining this package.

emacs-jupyter has proved so useful that I’ve been collecting a list of 
suggested improvements - I was actually intending on trying to contribute some 
of these myself, but I think it would be good to discuss first with you and 
other users of emacs-jupyter (is there an emacs-jupyter mailing list or 
preferred venue?). For now I’ll just share what I think would be the biggest 
improvement for my use case.

I use emacs-jupyter to connect from my laptop to remote jupyter kernels and 
notebook servers running on powerful machines provided by my employer. I work 
in a notebook locally on my laptop, but execute code blocks remotely. I specify 
a single remote jupyter notebook server+kernel to connect to in a #+PROPERTY 
line at the top of my org file which globally sets header-args for 
jupyter-python to include the correct :session argument.

For me, the biggest improvements would be addressing some pain points with 
disconnecting/reconnecting to the jupyter notebook server or kernel. I don’t 
have technical suggestions yet on how these could be addressed, so I’ll just 
describe the user experience side of things.

1) Reconnecting to the notebook server after disconnecting. Commonly I’ll work 
in my notebook, then take a break and close my laptop or disconnect from my 
work VPN. When I re-open my laptop/reconnect to VPN it seems like the 
connections to the remote notebook server are in a bad state (this may be a 
tramp thing, also). Currently my work around is to kill the associated emacs 
jupyter REPL buffer, then navigate to one of the source blocks in the notebook 
and run org-babel-pop-to-session which has the side-effect of re-connecting to 
the jupyter notebook server and recreating the jupyter REPL buffer. 

The main downside to this workaround is that the scroll back history of the 
REPL buffer is lost, since the buffer was killed. It would be great to be able 
to re-establish the connection to the REPL without discarding the buffer.

2) Getting results from a running block after disconnecting/reconnecting. 
Currently it’s difficult to manage long-running code blocks using emacs-jupyter 
if I disconnect from the remote server while the code block is running. The 
work around in #1 above to re-connect to a remote kernel doesn’t work if the 
kernel is still executing a block. If the kernel has completed executing the 
block, then the results are not populated back into the notebook (the execution 
UUID populated when the code block began executing remains).

I don’t know much about jupyter, but I saw some jupyter logs about buffered 
messages. Perhaps jupyter buffers output when a remote client disconnects? If 
so, maybe this buffered output could be replayed in a notebook when the emacs 
jupyter REPL connection is re-established?

3) Similarly, improving feedback from a long-running block would be very 
helpful. Currently you can use print statements in a long running block to 
report progress, but these are lost after disconnecting (maybe they are 
buffered on the jupyter server side and could be replayed on reconnect?). 
Further, I don’t know if this is feasible, but it would be amazing to have 
support for progress bars for long-running tasks (e.g., have a tqdm progress 
bar render in the org notebook).

Thanks again for the wonderful package - I hope we can talk a bit more about 
some of these friction points. I’d be happy to give a crack at contributing a 
PR, if that would be welcome.

Thank you,
David

> On Jan 4, 2022, at 15:24, Nathaniel Nicandro <nathanielnicandro@gmail.com> 
> wrote:
> 
> 
> Hello all,
> 
> I'm the maintainer of the emacs-jupyter project
> (https://github.com/nnicandro/emacs-jupyter) which essentially
> integrates Jupyter kernels (https://jupyter.org) with Org mode source
> blocks.
> 
> I wanted to make an introduction to the Org community.  So...hello!  And
> thanks for promoting the project on https://orgmode.org/features.html.
> 
> I believe a lot of users of the project use it mainly for the Org
> integration.  I thought it would be a good idea to get some feedback
> from the community on how their experience using emacs-jupyter has been.
> I'm getting back into active maintenance of the project and am looking
> for feedback to get a better idea of what the future of the project
> could look like.  What features of standard Org source blocks do you
> find Jupyter source blocks are lacking?  What potential features do you
> think would be useful for Jupyter source blocks to support, given the
> capabilities of Jupyter?  What would it mean to see Emacs-Jupyter and
> Org more integrated?  Of course, any other thoughts are welcome.
> 
> -- 
> Nathaniel
> 




reply via email to

[Prev in Thread] Current Thread [Next in Thread]