help-octave
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Slowness in function 'open'


From: Muthiah Annamalai
Subject: Re: Slowness in function 'open'
Date: Thu, 21 Jun 2007 15:08:53 -0500



On 6/21/07, Mark B. <address@hidden > wrote:

Hi,

Is there a faster way to load a big text file composed only of integers? I
am having severe performance penalties by using the load function

These are the results of my tests:

First test:
============================================
Octave, version 2.9.12 (configured for "i686-pc-msdosmsvc")

octave.exe:1> tic; load('mybigfile'); toc;
Elapsed time is 98.314560 seconds.

Matlab 7.3.0.267 (R2006b)

>> tic; load('mybigfile');toc;
Elapsed time is 27.923677 seconds.

Octave, version 2.1.73 (i686-pc-cygwin)

octave:1> tic; load('mybigfile');toc
ans = 1117.5
============================================

Between calls to the 'load' function I flushed the cache of the location
where the file resides

Second test:
============================================
octave.exe:2> tic; load('mybigfile'); toc;
Elapsed time is 93.594170 seconds.

>> tic; load('mybigfile');toc;
Elapsed time is 28.757878 seconds.
============================================

(Note: I cannot save the file in binary format because of incompatibility
with the C++ program that generates the text file)

Is this a known issue?



Update your Octave version if possible. I think you should do the Octave developers a favor, by using a newer version of Octave 2.9.x or later. The comparison with Matlab, doesnt make much difference now that we are on a much improved version.

I would also suggest you do a offline conversion program in Octave, that can load the text
files and save it to binary format using, 
save -binary cCaMeLCaPsMwItHuNgArIaNoTaTiOn

and then load binary files from Octave, in your 'main' script.

For me on the oldest Octave version I could find access to, 2.1.50
a 1.4MB file loads as text in 0.399s; whereas the same file loaded as binary
takes just 0.015370s.

I think thats x26 improvement for me.

What do you say?
-Muthu
PS: Im sorry if I posted this many times, just screwed it up



reply via email to

[Prev in Thread] Current Thread [Next in Thread]