Re: [O] org tables into R?

[Rainer M Krug]

Andreas Leha <address@hidden> writes:

> Hi Rainer,
>
> Thanks for chiming in.

Hope I can help...

>
> Rainer M Krug <address@hidden> writes:
>> Andreas Leha <address@hidden> writes:
>>
>>> Hi Aaron and Nicolas,
>>
>> Sorry for coming so late to this topic - I should try to figure out what
>> is causing the problem as I am the one responsible for this code...
>>
>>>
>>> Thanks for partly fixing this issue.  Unfortunately, when working with
>>> source block interactively the issue persists for me.  If in the same
>>> sample file I view the source block via 'C-c C-v v' and step through the
>>> generated code line-by-line, the table is again not split into columns.
>>>
>>> This time I immediately tried with 'emacs -Q' and I can reproduce the
>>> issue.
>>>
>>> Here is the ecm again:
>>>
>>> * test
>>> #+NAME: data
>>> |   A |  B |  C |
>>> |-----+----+----|
>>> | 115 | 76 | 60 |
>>> | 124 | 78 | 55 |
>>> | 118 | 73 | 65 |
>>> | 114 | 75 | 61 |
>>> | 108 | 74 | 82 |
>>>
>>> and pass it into R like this, then evaluation (C-c C-c) works, but
>>> stepping through the code generated for viewing (C-c C-v v) does not:
>>>
>>> #+BEGIN_SRC R :results output :exports both :session :var data.table=data
>>>   names(data.table)
>>>   head(data.table)
>>> #+END_SRC
>>>
>>> #+results:
>>> : [1] "A" "B" "C"
>>> :     A  B  C
>>> : 1 115 76 60
>>> : 2 124 78 55
>>> : 3 118 73 65
>>> : 4 114 75 61
>>> : 5 108 74 82
>>>
>>> #+begin_src emacs-lisp
>>>   (concat
>>>    (replace-regexp-in-string " on .*" "" (emacs-version))
>>>    "\n"
>>>    (replace-regexp-in-string " @.*" ")" (org-version nil t))
>>>    "\n"
>>>    (replace-regexp-in-string "].*" "]" (ess-version)))
>>> #+end_src
>>>
>>> #+results:
>>> : GNU Emacs 24.4.50.1 (x86_64-apple-darwin13.3.0, NS appkit-1265.21 Version 
>>> 10.9.4 (Build 13E28))
>>> :  of 2014-09-02
>>> : Org-mode version 8.3beta (release_8.3beta-717-ga8096c)
>>> : ess-version: 14.05 [git: 4283f1304a54502c42707b6a4ba347703f0992dd]
>>>
>>
>> ,----
>> | #+RESULTS:
>> | : GNU Emacs 24.4.1 (x86_64-apple-darwin14.0.0, Carbon Version 157 AppKit 
>> 1343.16)
>> | :  of 2014-11-24
>> | : Org-mode version 8.3beta (release_8.3beta-695-g390015)
>> | : ess-version: 14.09 [svn: 6041 (2014-09-13)]
>> `----
>>
>>
>> I get the same in the preview, but it works for me. I do not use -q.
>>
>> ,----
>> | > data.table <- local({
>> | +      con <- textConnection(
>> | +        "\"A\"    \"B\"   \"C\"
>> | + \"115\"  \"76\"  \"60\"
>> | + \"124\"  \"78\"  \"55\"
>> | + \"118\"  \"73\"  \"65\"
>> | + \"114\"  \"75\"  \"61\"
>> | + \"108\"  \"74\"  \"82\""
>> | +      )
>> | +      res <- utils::read.table(
>> | +        con,
>> | +        header    = TRUE,
>> | +        row.names = NULL,
>> | +        sep       = "\t",
>> | +        as.is     = TRUE
>> | +      )
>> | +      close(con)
>> | +      res
>> | +    })
>> | >   names(data.table)
>> | [1] "A" "B" "C"
>> | >   head(data.table)
>> |     A  B  C
>> | 1 115 76 60
>> | 2 124 78 55
>> | 3 118 73 65
>> | 4 114 75 61
>> | 5 108 74 82
>> | > 
>> `----
>>
>>
>>>
>>> Here is what I see in the org babel preview:
>>>
>>
>> This might be due to temail, but you don't have tabs (\t) between your
>> entries below - could you verify if this is the case? If yes, this might
>> be a locale issue (would be strange)?
>
> Yes, there seem to be no tabs.  If there were tabs, R should parse the
> table correctly, I guess.

Exactly. So somehow the tabs get replaced in your system by spaces.

>
> (BTW: My custom filter replacing tabs with spaces is still disabled,
> even though I pulled from master, which includes the recent
>
> commit 70f965535b07c03a40792237de3a970d1cb57551
> Author: Nicolas Goaziou <address@hidden>
> Date:   Mon Jan 5 12:39:26 2015 +0100
>
>     org-table: Ignore filters and hooks in radio tables
> )
>
>
>>
>>> data.table <- local({
>>>      con <- textConnection(
>>>        "\"A\"   \"B\"   \"C\"
>>> \"115\" \"76\"  \"60\"
>>> \"124\" \"78\"  \"55\"
>>> \"118\" \"73\"  \"65\"
>>> \"114\" \"75\"  \"61\"
>>> \"108\" \"74\"  \"82\""
>>>      )
>>>      res <- utils::read.table(
>>>        con,
>>>        header    = TRUE,
>>>        row.names = NULL,
>>>        sep       = "\t",
>>>        as.is     = TRUE
>>>      )
>>>      close(con)
>>>      res
>>>    })
>>> names(data.table)
>>> head(data.table)
>>
>> One solution would be to insert (instead of the tab character) "\t" to
>> separate the fields?
>>
>> You could try and modify the ob-R-transfer-variable-table-with-header -
>> From the help:
>>
>> ,----
>> | ob-R-transfer-variable-table-with-header is a variable defined in 
>> `ob-R.el'.
>> | Its value is
>> | "%s <- local({\n con <- textConnection(\n %S\n )\n res <-
>> | utils::read.table(\n con,\n header = %s,\n row.names = %s,\n sep =
>> | \"\\t\",\n as.is = TRUE\n )\n close(con)\n res\n })"
>> | 
>> |   This variable may be risky if used as a file-local variable.
>> | 
>> | Documentation:
>> | R code used to transfer a table defined as a variable from org to R.
>> | 
>> | This function is used when the table contains a header.
>> `----
>>
>> This is passed to the (format) function and one might be able to change
>> something there.
>
> Thanks for the hint.  Do I assume correctly, that this variable is used
> when the code block is evaluated as well?  In that case, I am reluctant
> to change it, as the evaluation now works.  The surprising thing is the
> different behaviour for preview and evaluation.

Yup - you are right. These variables (ob-R-transfer-variable-table-with-header 
and
ob-R-transfer-variable-table-without-header) are only used in the
function org-babel-R-assign-elisp which is only called from
org-babel-variable-assignments:R which is doing the variable assignments
whenever they are needed.

What would definitely solve the issue is if the string containing the
tabs would use \t instead - but I have no idea how this could be
achieved easily.

>
> I am extremely tight with my time right now, so that I won't be able to
> do much experimentation/investigation right now, I am afraid.  But I
> will try to still be helpful...

The question is really "where does the tab become a space". As you see
the spaces after C-c C-v v it seems that the problem also occurs when
the function  org-babel-expand-src-block is called. So I think to narrow
down what is happening, you have to step through this function and see
if the assignments contain spaces instead of tabs from the beginning or
if they are changed whithin the function.

As only you see these errors, I would still guess that it is something
with your system (locale?, utf et al issues?).

By the way - are you starting R as --vanilla? If not, it might be worth
trying it out?

,----
|   (setq inferior-R-args "--vanilla")
`----

Actually, if you don't see tabs after calling org-babel-expand-src-block
on the example block, this is a pure org issue - no ESS or R involved here.

One thing you could try is to change the sep value in the variables
ob-R-transfer-variable-table-with-header and
ob-R-transfer-variable-table-without-header as follow:

,----
|   ...  
|        sep       = \"\",   ;;;;;<<<<<<<<< was before  sep = \"\\t\"
|   ...
`----

This will set the separator to "whitespace", i.e. "that is one or more
spaces, tabs, newlines or carriage returns" (from R help).

This should solve your problem.

Let us know if this solves your problem.

Cheers,

Rianer

>
> Thanks,
> Andreas
>
>
>>
>> Cheers,
>>
>> Rainer
>>
>>>
>>>
>>> And finally, here is my R session, when I step through the code:
>>>
>>>
>>> R version 3.1.2 (2014-10-31) -- "Pumpkin Helmet"
>>> Copyright (C) 2014 The R Foundation for Statistical Computing
>>> Platform: x86_64-apple-darwin10.8.0 (64-bit)
>>>
>>> R is free software and comes with ABSOLUTELY NO WARRANTY.
>>> You are welcome to redistribute it under certain conditions.
>>> Type 'license()' or 'licence()' for distribution details.
>>>
>>>   Natural language support but running in an English locale
>>>
>>> R is a collaborative project with many contributors.
>>> Type 'contributors()' for more information and
>>> 'citation()' on how to cite R or R packages in publications.
>>>
>>> Type 'demo()' for some demos, 'help()' for on-line help, or
>>> 'help.start()' for an HTML browser interface to help.
>>> Type 'q()' to quit R.
>>>
>>>> > options(STERM='iESS', str.dendrogram.last="'", editor='emacsclient', 
>>>> > show.error.locations=TRUE)
>>>> data.table <- local({
>>> +      con <- textConnection(
>>> +        "\"A\"   \"B\"   \"C\"
>>> + \"115\" \"76\"  \"60\"
>>> + \"124\" \"78\"  \"55\"
>>> + \"118\" \"73\"  \"65\"
>>> + \"114\" \"75\"  \"61\"
>>> + \"108\" \"74\"  \"82\""
>>> +      )
>>> +      res <- utils::read.table(
>>> +        con,
>>> +        header    = TRUE,
>>> +        row.names = NULL,
>>> +        sep       = "\t",
>>> +        as.is     = TRUE
>>> +      )
>>> +      close(con)
>>> +      res
>>> +    })
>>>> names(data.table)
>>> [1] "A...B...C"
>>>> head(data.table)
>>>    A...B...C
>>> 1 115 76  60
>>> 2 124 78  55
>>> 3 118 73  65
>>> 4 114 75  61
>>> 5 108 74  82
>>>
>>>
>>> Regards,
>>> Andreas
>>>
>>>
>>>
>
>
>

-- 
Rainer M. Krug
email: Rainer<at>krugs<dot>de
PGP: 0x0F52F982

signature.asc
Description: PGP signature