Re: [Orgmode] Org-babel `:hlines yes` no longer working for python

[Eric Schulte]

Hi Chris,

Thanks for catching this.  I've just pushed up a patch which should fix
the issue.

Best -- Eric

Christopher Allan Webber <address@hidden> writes:

> Hm.  I've found a bug with this patch:
>
> #+begin_src python
> return [['foo', 'bar', 'baz'], ["a", "b", "None of the above"], ['1', 2, 3]]
> #+end_src
>
> #+results:
> | foo | bar | baz                |
> | a   | b   | hline of the above |
> | 1   | 2   | 3                  |
>
> This also replaces the word "None" anywhere with hline, even in
> strings.
>
>
> "Eric Schulte" <address@hidden> writes:
>
>> Hi,
>>
>> OK, I've applied this patch.
>>
>> Christopher Allan Webber <address@hidden> writes:
>>
>>> Eric,
>>>
>>> Looks good to me!  It's abusing the None type's meaning a little, but
>>> I think it's acceptable enough.  (If you think of hlines as rows that
>>> are not rows, you can trick yourself into thinking it is perfectly
>>> pythonic :))
>>>
>>
>> Yea, this semantic mismatch bothered me, however it looks like Python
>> doesn't have anything like symbols that could be used here, and I guess
>> there isn't an issue of wanting to preserve "None" for "nil" mapping
>> because "nil" can be represented with an empty list "[]".
>>
>> Thanks for bringing this up! -- Eric
>>
>>>
>>>  - cwebb
>>>
>>> "Eric Schulte" <address@hidden> writes:
>>>
>>>> Hi Christopher,
>>>>
>>>> I'm certainly no Python expert, but I implemented your idea of
>>>> converting "hlines" to and from "None"'s (patch below [1]), and it seems
>>>> to work (under some definition of work).  See the following example with
>>>> the new behavior.
>>>>
>>>>
>>>> --8<---------------cut here---------------start------------->8---
>>>> #+tblname: many-cols
>>>> | a | b | c |
>>>> |---+---+---|
>>>> | d | e | f |
>>>> |---+---+---|
>>>> | g | h | i |
>>>>
>>>> #+source: echo-table
>>>> #+begin_src python :var tab=many-cols :hlines yes
>>>>   return tab
>>>> #+end_src
>>>>
>>>> #+results: echo-table
>>>> | a | b | c |
>>>> |---+---+---|
>>>> | d | e | f |
>>>> |---+---+---|
>>>> | g | h | i |
>>>> --8<---------------cut here---------------end--------------->8---
>>>>
>>>> Please, Python people, try this out and if you like the behavior then
>>>> I'll happily apply the patch.
>>>>
>>>> Best -- Eric
>>>>
>>>> Christopher Allan Webber <address@hidden> writes:
>>>>
>>>>> Hey Eric,
>>>>>
>>>>> Thanks for the super helpful reply!
>>>>>
>>>>> Out of curiosity, is it likely that we will ever get hline support in
>>>>> Python and etc?  I've been pondering how it might be done, and maybe it
>>>>> could be like this, using a '|-' string instead of a list for the row:
>>>>>
>>>>> [['a', 'b', 'c'], '|-', ['d', 'e', 'f'], ['g', 'h', 'i']]
>>>>>
>>>>> Which would produce:
>>>>>
>>>>> | a | b | c |
>>>>> |---+---+---|
>>>>> | d | e | f |
>>>>> | g | h | i |
>>>>>
>>>>> Alternately maybe the same thing could be done by abusing None:
>>>>>
>>>>> [['a', 'b', 'c'], None, ['d', 'e', 'f'], ['g', 'h', 'i']]
>>>>>
>>>>> Thoughts?
>>>>>  - cwebb
>>>>>
>>>>> "Eric Schulte" <address@hidden> writes:
>>>>>
>>>>>> Hi Christopher,
>>>>>>
>>>>>> Thanks for pointing this out, this is an error in the documentation,
>>>>>> which I will update.  The code you posted should generate the error you
>>>>>> have received.
>>>>>>
>>>>>> Currently the only language which can handle hlines is emacs-lisp, all
>>>>>> other languages will result in errors like the one you pasted below.
>>>>>> That's not to say that it wouldn't be possible to add hline handling to
>>>>>> other languages, or to maybe do something tricky like session-based
>>>>>> evaluation in which an `hlines' variable was pre-initialized to some
>>>>>> value, but I digress.
>>>>>>
>>>>>> Note that it *is* possible to have hlines in the output, using colnames,
>>>>>> e.g.
>>>>>>
>>>>>>
>>>>>> --8<---------------cut here---------------start------------->8---
>>>>>> #+tblname: A
>>>>>> | a | b | c |
>>>>>> |---+---+---|
>>>>>> | d | e | f |
>>>>>> | g | h | i |
>>>>>>
>>>>>> #+begin_src python :var tab=A :colnames yes
>>>>>> return [[val + '*' for val in row] for row in tab]
>>>>>> #+end_src
>>>>>>
>>>>>> #+results:
>>>>>> | a  | b  | c  |
>>>>>> |----+----+----|
>>>>>> | d* | e* | f* |
>>>>>> | g* | h* | i* |
>>>>>> --8<---------------cut here---------------end--------------->8---
>>>>>>
>>>>>> which works because the hline, and the column names, are never made
>>>>>> available to python, rather Babel holds onto them and then re-applies
>>>>>> them to the source block's output.
>>>>>>
>>>>>> or even to have an elisp block add hlines to your results
>>>>>>
>>>>>>
>>>>>> --8<---------------cut here---------------start------------->8---
>>>>>> #+tblname: many-cols
>>>>>> | a | b | c |
>>>>>> |---+---+---|
>>>>>> | d | e | f |
>>>>>> |---+---+---|
>>>>>> | g | h | i |
>>>>>>
>>>>>> #+source: echo-table
>>>>>> #+begin_src python :var tab=many-cols
>>>>>>   return tab
>>>>>> #+end_src
>>>>>>
>>>>>> #+begin_src emacs-lisp :var table=echo-table
>>>>>>   (butlast (apply #'append (mapcar (lambda (el) (list el 'hline)) 
>>>>>> table)))
>>>>>> #+end_src
>>>>>>
>>>>>> #+results:
>>>>>> | a | b | c |
>>>>>> |---+---+---|
>>>>>> | d | e | f |
>>>>>> |---+---+---|
>>>>>> | g | h | i |
>>>>>> --8<---------------cut here---------------end--------------->8---
>>>>>>
>>>>>> Thanks for pointing this out!
>>>>>>
>>>>>> Best -- Eric
>>>>>>
>>>>>> Christopher Allan Webber <address@hidden> writes:
>>>>>>
>>>>>>> Hello all,
>>>>>>>
>>>>>>> I was going through the tutorial and testing the :hlines yes feature as
>>>>>>> described in the info manual.  Unfortunately, the example given no
>>>>>>> longer seems to work for python:
>>>>>>>
>>>>>>> #+tblname: many-cols
>>>>>>> | a | b | c |
>>>>>>> |---+---+---|
>>>>>>> | d | e | f |
>>>>>>> |---+---+---|
>>>>>>> | g | h | i |
>>>>>>>
>>>>>>> #+source: echo-table
>>>>>>> #+begin_src python :var tab=many-cols :hlines yes
>>>>>>>   return tab
>>>>>>> #+end_src
>>>>>>>
>>>>>>> #+results: echo-table
>>>>>>> | a | b | c |
>>>>>>> | d | e | f |
>>>>>>> | g | h | i |
>>>>>>>
>>>>>>> In the buffer *Org-Babel Error Output* I see:
>>>>>>>
>>>>>>> Traceback (most recent call last):
>>>>>>>   File "<stdin>", line 6, in <module>
>>>>>>>   File "<stdin>", line 3, in main
>>>>>>> NameError: global name 'hline' is not defined
>>>>>>>
>>>>>>> In emacs-lisp this still seems to work though.  But I also see that in
>>>>>>> emacs lisp hlines are represented by the hline symbol.  I'm guessing
>>>>>>> that the python equivalent was trying to do the same thing, but no hline
>>>>>>> variable exists in python?
>>>>>>>
>>>>>>> Thanks!
>>>>>>>  - cwebb
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> Emacs-orgmode mailing list
>>>>>>> Please use `Reply All' to send replies to the list.
>>>>>>> address@hidden
>>>>>>> http://lists.gnu.org/mailman/listinfo/emacs-orgmode
>>>>
>>>> Footnotes: 
>>>> [1]  
>>>>
>>>> diff --git a/lisp/babel/langs/ob-python.el b/lisp/babel/langs/ob-python.el
>>>> index 2ce9e1d..29bb166 100644
>>>> --- a/lisp/babel/langs/ob-python.el
>>>> +++ b/lisp/babel/langs/ob-python.el
>>>> @@ -96,7 +96,7 @@ called by `org-babel-execute-src-block'."
>>>>  specifying a var of the same value."
>>>>    (if (listp var)
>>>>        (concat "[" (mapconcat #'org-babel-python-var-to-python var ", ") 
>>>> "]")
>>>> -    (format "%S" var)))
>>>> +    (if (equal var 'hline) "None" (format "%S" var))))
>>>>  
>>>>  (defun org-babel-python-table-or-string (results)
>>>>    "If the results look like a list or tuple, then convert them into an
>>>> @@ -110,7 +110,9 @@ Emacs-lisp table, otherwise return the results as a 
>>>> string."
>>>>                   "\\[" "(" (replace-regexp-in-string
>>>>                              "\\]" ")" (replace-regexp-in-string
>>>>                                         ", " " " (replace-regexp-in-string
>>>> -                                                 "'" "\"" results))))))
>>>> +                                                 "'" "\""
>>>> +                                           (replace-regexp-in-string
>>>> +                                            "None" "hline" results 
>>>> t)))))))
>>>>       results)))
>>>>  
>>>>  (defvar org-babel-python-buffers '(:default . nil))