Re: [O] Export arrays for 'sh' code blocks when using bash

[Eric Schulte]

Hi Pascal,

This looks like a good patch.  If I could make three changes/requests.

1. This patch should also work for "begin_src bash" code blocks.  After
   a very quick glance it doesn't appear to.

2. Could you include one or two tests ensuring that this patch does what
   it intends with bash code blocks, and does not affect code blocks
   executed with bin/sh?

3. Large contributions like this require some FSF paperwork.  Please see
   the following page for information on requirements to contribute.

   http://orgmode.org/worg/org-contribute.html

Thanks, and I look forward to seeing this merged into Org-mode!
Eric

Pascal Fleury <address@hidden> writes:

> Hello,
>
> I'dl like to propose a patch for inclusion into org-mode (ob-shell.el in
> particular).
>
> *TL;DR:* use arrays and associative arrays when exporting variables to bash
> in 'sh' code blocks.
>
> *Details:*
> When variables are defined in a 'sh' code block, they are exported as
> strings. when the variable itself is an array or a table, then we simply
> get a shell variable that contains the list of all values in a
> non-structured form. E.g.
>
> #+NAME: my_list
> | one   |
> | two   |
> | three |
>
> #+NAME: experiment
> | name | first_attempt    |
> | date | [2014-03-27 Thu] |
> | user | fleury           |
>
> #+BEGIN_SRC sh :var scalar="some value" :var array=my_list :var table=config
> echo ${scalar}  # -> prints 'some value'
> echo ${array}   # -> prints 'one two three'
> echo ${table}   # -> prints 'first attempt [2014-03-27 Thu] fleury'
> #+END_SRC
>
> This will print simple strings. Also, there is no easy way to access the
> date of the experiment, for example. Now bash has things like arrays and
> associative arrays, but the current ob-shell.el does not use these.
> Probably because their syntax is bash-specific, and ob-shell.el is
> shell-agnostic.
>
> My patch (attached) changes this in the case you have
> (setq org-babel-sh-command "bash")
> in your emacs config somewhere. If any other value is used, it continues to
> export them as we do today (I don't know enough about other shells).
>
> In that case, it will export the list as an array, the table as an
> associative array, and the scalar as it does already. So the 'sh' code
> block could then use
>
> #+BEGIN_SRC sh :var scalar="some value" :var array=my_list :var table=config
> echo ${scalar}
> echo ${array[1]} # -> prints "two"
> echo ${table[user]} # -> prints "fleury"
> #+END_SRC
>
> In the case we have a bigger table, then the first row is used as key, the
> others are represented as a string with all the values (as it is for array
> currently). bash does not have multi-dimensional arrays, so it needs to be
> a string.
>
> This makes writing shell snippets much easier in my experience, and there
> I'd like to propose this fix for the org-mode community at large.
>
> --paf
>
> diff --git a/lisp/ob-shell.el b/lisp/ob-shell.el
> index 3ede701..0a691e2 100644
> --- a/lisp/ob-shell.el
> +++ b/lisp/ob-shell.el
> @@ -106,6 +106,30 @@ This function is called by 
> `org-babel-execute-src-block'."
>  
>  ;; helper functions
>  
> +(defun org-babel-variable-assignments:bash_array (varname values &optional 
> sep hline)
> +  "Returns a list of statements declaring the values as a bash array."
> +  (format "declare -a %s=( \"%s\" )"
> +     varname
> +     (mapconcat 'identity
> +       (mapcar
> +         (lambda (value) (org-babel-sh-var-to-sh value sep hline))
> +         values)
> +       "\" \"")))
> +
> +(defun org-babel-variable-assignments:bash_associative (varname values 
> &optional sep hline)
> +  "Returns a list of statements declaring the values as bash associative 
> array."
> +  (format "declare -A %s\n%s"
> +    varname
> +    (mapconcat 'identity
> +      (mapcar
> +        (lambda (items)
> +          (format "%s[\"%s\"]=%s"
> +            varname
> +            (org-babel-sh-var-to-sh (car items) sep hline)
> +            (org-babel-sh-var-to-sh (cdr items) sep hline)))
> +        values)
> +      "\n")))
> +
>  (defun org-babel-variable-assignments:sh (params)
>    "Return list of shell statements assigning the block's variables."
>    (let ((sep (cdr (assoc :separator params)))
> @@ -114,9 +138,17 @@ This function is called by 
> `org-babel-execute-src-block'."
>                    "hline"))))
>      (mapcar
>       (lambda (pair)
> -       (format "%s=%s"
> -            (car pair)
> -            (org-babel-sh-var-to-sh (cdr pair) sep hline)))
> +       (if (and (string= org-babel-sh-command "bash") (listp (cdr pair)))
> +         (if (listp (car (cdr pair)))
> +           (org-babel-variable-assignments:bash_associative
> +           (car pair) (cdr pair) sep hline)
> +           (org-babel-variable-assignments:bash_array
> +           (car pair) (cdr pair) sep) hline)
> +         (format "%s=%s"
> +         (car pair)
> +         (org-babel-sh-var-to-sh (cdr pair) sep hline))
> +       )
> +     )
>       (mapcar #'cdr (org-babel-get-header params :var)))))
>  
>  (defun org-babel-sh-var-to-sh (var &optional sep hline)

-- 
Eric Schulte
https://cs.unm.edu/~eschulte
PGP: 0x614CA05D