Re: [O] ipython integration fails

[John Kitchin]

I guess you are having a problem with the version of jupyter, and maybe
jupyter_client. Maybe upgrading thises will fix the ipython problem.

See notes below for python sessions.
Prof. Dr. Johanna May <address@hidden> writes:

> Hi everybody,
>
> I am creating lecture notes including some python snippets. Now, in
> order to teach jupyter notebook use I would like to include just the
> same code as in a python snippet in an ipython snippet.

If you get ipython working, then you can write your lectures in
org-mode, and use ox-ipynb from scimax to export a jupyter notebook from
the org-file.
>
> However ipython blocks don't work in my set up and I could not figure
> out which dependency or config line I might need to add in order to do
> this properly (and scimax also creates errors, but since I have my own
> config running I thought ipython might  already do the job).
>
> The ipython snippet I used is:
> #+BEGIN_SRC ipython :session :results raw drawer
>   %matplotlib inline
>   import matplotlib.pyplot as plt
>   import numpy as np
> #+END_SRC
>
> The error message is:
> Traceback (most recent call last):
>   File "/home/username/.emacs.d/elpa/ob-ipython-20180224.953/client.py", line 
> 60, in <module>
>     c = create_client(args.conn_file)
>   File "/home/username/.emacs.d/elpa/ob-ipython-20180224.953/client.py", line 
> 43, in create_client
>     cf = find_connection_file('emacs-' + name)
>   File "/usr/lib/python3.6/site-packages/jupyter_client/connect.py", line 
> 218, in find_connection_file
>     raise IOError("Could not find %r in %r" % (filename, path))
> OSError: Could not find 'emacs-default' in ['.', '/run/user/1000/jupyter']
>
> Maybe I do not need ipython anyway. What works are python code snippets
> like the following one. Minted exports both, code and results and they
> are both displayed in the pdf slides/manuscript (using lualatex,
> beamerarticle etc.)
>
> #+begin_src python :results output :session :exports both
> import time
> time.sleep(3)
> #+end_src
>
> However, then, what does not work properly is several code blocks in one
> session, e.g.
>
> (1)
> #+begin_src python :results output :session :exports both
> import pandas as pd
> import numpy as np
> import matplotlib.pyplot as plt
> import datetime as dt
> import seaborn as sns
> sns.set(style="darkgrid")
> #+end_src
>
> (2)
> #+begin_src python :results output :session :exports both
> df = pd.read_csv('./household_data_1min_singleindex.csv')
> #+end_src
>
> (3)
> #+begin_src python :results output :session :exports both
> zeit=df.utc_timestamp.apply(lambda x: dt.datetime.strptime(x, 
> "%Y-%m-%dT%H:%M:%SZ"))
> #+end_src
>
> (4)
> #+begin_src python :results file :session :var 
> matplot_lib_filename=(org-babel-temp-file "figu" ".png"),fontsize=fs :exports 
> both
> plt.rcParams.update({'font.size':fontsize})
> rcParams.update({'figure.autolayout': True})
> plt.plot(zeit,df.DE_KN_residential2_washing_machine)
> plt.savefig(matplot_lib_filename)
> matplot_lib_filename
> #+end_src

To get this to work, I used

#+begin_src python :results output :session :exports both
import time
time.sleep(3)
#+end_src

#+RESULTS:
: Python 3.6.0 |Anaconda custom (x86_64)| (default, Dec 23 2016, 13:19:00)
: [GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.57)] on darwin
: Type "help", "copyright", "credits" or "license" for more information.
: >>> python.el: native completion setup loaded

However, then, what does not work properly is several code blocks in one
session, e.g.

(1)
#+begin_src python :results output :session :exports both
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import datetime as dt
import seaborn as sns
sns.set(style="darkgrid")
#+end_src

#+RESULTS:

(2)
#+begin_src python :results output :session :exports both
x = np.linspace(0, 1)
y = x**2
#+end_src

#+RESULTS:

(3)
#+begin_src python :results output :session :exports both
fontsize=12
#+end_src


(4)
#+begin_src python :results file :session :var 
matplot_lib_filename=(org-babel-temp-file "figu" ".png") :exports both
plt.rcParams.update({'font.size':fontsize})
plt.rcParams.update({'figure.autolayout': True})
plt.plot(x, y)
plt.savefig(matplot_lib_filename)
print(matplot_lib_filename)
#+end_src

5)

#+begin_src python :results file :session :var 
matplot_lib_filename=(org-babel-temp-file "figu" ".png") :exports both
plt.figure()
plt.rcParams.update({'font.size':fontsize})
plt.rcParams.update({'figure.autolayout': True})
plt.plot(x, 2 * x)
plt.savefig(matplot_lib_filename)
print(matplot_lib_filename)
#+end_src

which for me worked like I think you expect, which is your run each
block sequentially. There were some errors in your last block related to
the fontsize and rcParams that are different in my block. Probably I
have different defaults for the results (I always use output) so I
printed the results instead of using the returned value.
>
> I do not like the fact that plt.rcParams.update and other lines that
> jupyter does not need get exported - is there a simple way to exclude
> certain lines from export?

I don't think so. If you want the Jupyter plots to look like these, it
seems like you need to include them. Otherwise, you have to put the
lines in a separate block, with :exports none in the header.
>
> And I do get in serious trouble when I add a second data set (another csv) 
> and do
> further analysis steps - all graphics then look the same and bad and do
> not represent the actual result of the code block. It seems that all get
> the same wrong name and then at each graphics location this same png
> gets displayed.

This isn't something I was able to reproduce. I get different temp file
names with the code above.
>
> So, maybe there is a good tutorial on using sessions in ob-python or
> some other hint that I could use?
>
> Is there a way to restart the (python) kernel with some header message
> in the src block?

In scimax you can do this on an ipython src block with :restart in the
header. You can restart the Python session by killing the *Python*
buffer.

I think you can get something like this in a header with:
#+header: :var restart=(when (get-buffer "*Python*") (kill-buffer "*Python*") 0)

>
> Thank you very much,
>
> Cheers,
>
> J. May


--
Professor John Kitchin
Doherty Hall A207F
Department of Chemical Engineering
Carnegie Mellon University
Pittsburgh, PA 15213
412-268-7803
@johnkitchin
http://kitchingroup.cheme.cmu.edu