help-octave
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Is there something quicker than textscan?


From: Reza Housseini
Subject: Re: Is there something quicker than textscan?
Date: Mon, 19 Mar 2012 13:35:07 +0100



On Mon, Mar 19, 2012 at 12:44 PM, PhilipNienhuis <address@hidden> wrote:

maiky76 wrote
>
> Hi,
>
> I have to load quite a few files from an analyzer (.txt with 4 columns of
> 16384 lines of fft: freq, Gain, Phase and Coherence with plus a footer
> with the analyzer settings that i don't care about).
>
> I use the following code in a loop to store the data before performing
> some calculation in the loop:
>
> [...]
>
> fid   = fopen(D_FR(k).name);
> data  = textscan(fid,'%f %f %f %f',16384);
> fclose(fid);
>
> FRTF_gain(:,k)        = data{1,2};
> FRTF_phase(:,k)  = data{1,3};
> FRTF_coh(:,k)     = data{1,4};
>
> [...]
>
> The slowest step by (very) far is the textscan process.
> I average 85s per file with Octave 3.6.1 and 0.25s with Matlab 2010a.
>

Yeah, textread and textscan have a pathetic performance when a format repeat
count is specified on relatively long files. The cause lies in about the
only piece of original code I haven't touched yet.

I have a patch in mind (and tried in a debug situation) that will
significantly speed up your problem (though not as fast as ML), but that
will have to wait until (1) I got time to implement it and (2) release of
Octave 3.8. Or maybe 3.6.2 if it the core developers agree that the poor
performance is a bug (I'd say so but who am I).

This situation will only get better as soon as a binary textscan has been
implemented. It is in the making but it will probably take some more time.



> Question: how could I load the data quicker than that in Octave?
>

As I answered a few days go in a thread with a similar subject, Octave
offers many text I/O solutions, not just textscan.
In your case:

C = dlmread (D_FR(k).name, ' ', 'A1:D16384')

...probably works fine.

(see 'help dlmread')

In a general sense, don't complain too soon that "ML does it
better/faster/....". That may be true for that specific literal piece of ML
code, but depending on the situation Octave offers alternative solutions may
be on par or sometimes even better.

Philip


--
View this message in context: http://octave.1599824.n4.nabble.com/Is-there-something-quicker-than-textscan-tp4483991p4484467.html
Sent from the Octave - General mailing list archive at Nabble.com.
_______________________________________________
Help-octave mailing list
address@hidden
https://mailman.cae.wisc.edu/listinfo/help-octave

I used dlmread for files with more then 35000 lines. There's no remarkable delay and it fits perfectly for your purpose.

reply via email to

[Prev in Thread] Current Thread [Next in Thread]