[Top][All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [O] Bug? R: Org babel block execution *drastically* slower than in E
From: |
Nick Dokos |
Subject: |
Re: [O] Bug? R: Org babel block execution *drastically* slower than in ESS session directly |
Date: |
Wed, 31 Oct 2012 19:00:20 -0400 |
Thomas S. Dye <address@hidden> wrote:
> Hi John,
>
> Have you tried wrapping your R read in system.time()? If you are right
> about :results silent eating up lots of time, then this should fix the
> problem. system.time yields just a bit of output, so shouldn't slow
> things down if writing out the data is indeed the problem as you
> suspect.
>
I missed this in the thread so I thought I'd add one more thing to try:
o run iostat (or equivalent) to check how your disks are doing.
on linux, iostat is part of a package usually called sysstat.
Nick
>
>
> John Hendy <address@hidden> writes:
>
> > On Wed, Oct 31, 2012 at 3:12 PM, <address@hidden> wrote:
> >
> >> John Hendy <address@hidden> writes:
> >>
> >> > On Wed, Oct 31, 2012 at 11:41 AM, <span dir="ltr"><mailto:
> >> address@hidden></span> wrote:
> >> > John Hendy <mailto:address@hidden> writes:
> >> >
> >> >> I edited the subject to be more concise/clear.I let orgmode chug away
> >> >> on reading in some ~10-30mb csv files for nearly 30min.
> >> >
> >> > [rest deleted]
> >> >
> >> > You need an ECM.I did my best to provide one, other than the file, which
> >> I offered to provide if others requested that I upload it somewhere. Since
> >> you have done so, so have I:
> >> > - https://docs.google.com/open?id=0BzQupOSnvw08WHdabHh5VVczRGM
> >>
> >> > Let me know if that doesn't work. I put it on Google docs and
> >> sometimes have issues with the sharing settings...
> >>
> >> Not an ECM in my book, but ...
> >>
> >>
> > What else would you like? I provided:
> > - the config
> > - the data
> > - how to [attempt to] reproduce
> > - the org-mode text
> >
> >
> >
> >> On my 4 year old MacBook:
> >>
> >> ,----
> >> |
> >> | #+PROPERTY: session *R*
> >> |
> >> | #+name: bigcsv
> >> | #+begin_src R
> >> | bigcsv <- Sys.glob("~/Downloads/*.csv")
> >> | #+end_src
> >> |
> >> | #+RESULTS: bigcsv
> >> | : /Users/cberry/Downloads/test-file.csv
> >> |
> >> | #+name: readbig
> >> | #+begin_src R :results output
> >> | system.time(
> >> | tmp <- read.csv(bigcsv)
> >> | )
> >> |
> >> | #+end_src
> >> |
> >> | #+RESULTS: readbig
> >> | : user system elapsed
> >> | : 5.679 0.306 6.002
> >> |
> >> `----
> >>
> >> About the same as running from ESS.
> >>
> >>
> > Not sure what to say. Looking for ways to troubleshoot or confirm. Since
> > you can't confirm, any suggestions on where I should look for my issue? I
> > can't explain it! All I know is that org chugs and chugs and the direct
> > execution in ESS session is lightning fast.
> >
> >
> >
> > FYI,
> >>
> >> > dim(tmp)
> >> [1] 964805 9
> >>
> >>
> >>
> > Was that just to show the size, or is something wrong with my file?
> >
> >
> >> Chuck
> >>
> >>
> >
> > Thanks for attempting to replicate!
> > John
> >
> >
> >> [ rest deleted ]
> >>
> >>
> >>
> >>
> > On Wed, Oct 31, 2012 at 3:12 PM, <address@hidden> wrote:
> >
> > John Hendy <address@hidden> writes:
> >
> > > On Wed, Oct 31, 2012 at 11:41 AM, <span
> > dir="ltr"><mailto:address@hidden></span> wrote:
> > > John Hendy <mailto:address@hidden> writes:
> > >
> > >> I edited the subject to be more concise/clear.I let orgmode
> > chug away
> > >> on reading in some ~10-30mb csv files for nearly 30min.
> > >
> > > [rest deleted]
> > >
> > > You need an ECM.I did my best to provide one, other than the
> > file, which I offered to provide if others requested that I upload
> > it somewhere. Since you have done so, so have I:
> > > - https://docs.google.com/open?id=0BzQupOSnvw08WHdabHh5VVczRGM
> >
> > > Let me know if that doesn't work. I put it on Google docs
> > and sometimes have issues with the sharing settings...
> >
> > Not an ECM in my book, but ...
> >
> >
> >
> > What else would you like? I provided:
> > - the config
> > - the data
> > - how to [attempt to] reproduce
> > - the org-mode text
> >
> >
> >
> > On my 4 year old MacBook:
> >
> > ,----
> > |
> > | #+PROPERTY: session *R*
> > |
> > | #+name: bigcsv
> > | #+begin_src R
> > | bigcsv <- Sys.glob("~/Downloads/*.csv")
> > | #+end_src
> > |
> > | #+RESULTS: bigcsv
> > | : /Users/cberry/Downloads/test-file.csv
> > |
> > | #+name: readbig
> > | #+begin_src R :results output
> > | system.time(
> > | tmp <- read.csv(bigcsv)
> > | )
> > |
> > | #+end_src
> > |
> > | #+RESULTS: readbig
> > | : user system elapsed
> > | : 5.679 0.306 6.002
> > |
> > `----
> >
> > About the same as running from ESS.
> >
> >
> >
> > Not sure what to say. Looking for ways to troubleshoot or confirm.
> > Since you can't confirm, any suggestions on where I should look for my
> > issue? I can't explain it! All I know is that org chugs and chugs and
> > the direct execution in ESS session is lightning fast.
> >
> >
> >
> > FYI,
> >
> > > dim(tmp)
> > [1] 964805 9
> >
> >
> >
> >
> > Was that just to show the size, or is something wrong with my file?
> >
> >
> > Chuck
> >
> >
> >
> >
> > Thanks for attempting to replicate!
> > John
> >
> >
> > [ rest deleted ]
> >
> >
> >
> >
> >
>
> --
> Thomas S. Dye
> http://www.tsdye.com
>