Re: Is there something quicker than textscan?

[PhilipNienhuis]

maiky76 wrote
> 
> Hi,
> 
> I have to load quite a few files from an analyzer (.txt with 4 columns of 
> 16384 lines of fft: freq, Gain, Phase and Coherence with plus a footer
> with the analyzer settings that i don't care about).
> 
> I use the following code in a loop to store the data before performing
> some calculation in the loop:
> 
> [...]
> 
> fid   = fopen(D_FR(k).name);
> data  = textscan(fid,'%f %f %f %f',16384); 
> fclose(fid);
> 
> FRTF_gain(:,k)        = data{1,2};
> FRTF_phase(:,k)  = data{1,3};
> FRTF_coh(:,k)     = data{1,4};
> 
> [...]
> 
> The slowest step by (very) far is the textscan process. 
> I average 85s per file with Octave 3.6.1 and 0.25s with Matlab 2010a.
> 

Yeah, textread and textscan have a pathetic performance when a format repeat
count is specified on relatively long files. The cause lies in about the
only piece of original code I haven't touched yet.

I have a patch in mind (and tried in a debug situation) that will
significantly speed up your problem (though not as fast as ML), but that
will have to wait until (1) I got time to implement it and (2) release of
Octave 3.8. Or maybe 3.6.2 if it the core developers agree that the poor
performance is a bug (I'd say so but who am I).

This situation will only get better as soon as a binary textscan has been
implemented. It is in the making but it will probably take some more time.



> Question: how could I load the data quicker than that in Octave?
> 

As I answered a few days go in a thread with a similar subject, Octave
offers many text I/O solutions, not just textscan.
In your case:

C = dlmread (D_FR(k).name, ' ', 'A1:D16384')

...probably works fine.

(see 'help dlmread')

In a general sense, don't complain too soon that "ML does it
better/faster/....". That may be true for that specific literal piece of ML
code, but depending on the situation Octave offers alternative solutions may
be on par or sometimes even better.

Philip


--
View this message in context: 
http://octave.1599824.n4.nabble.com/Is-there-something-quicker-than-textscan-tp4483991p4484467.html
Sent from the Octave - General mailing list archive at Nabble.com.