Re: reading text file with textscan annoyingly slow

[MarcelK]

Well, I've installed Octave 3.4.3. and it works just fine.
Also, it works with GUIOctave if I use gnuplot as graphics toolkit.

There's even an improvement in speed when using textscan but it still takes
about 3 seconds to read one single file, which is unfortunately still too
slow for my purposes.

My intention is to read 365 of these files into a cell array,so the user
would have to wait ~20 minutes until the data is read, which is quite too
much I think.

I tried to write another function which reads the data faster but the result
is just slightly better.
Maybe somebody has some advice for me how to speed it up.
Here's the code:


function [Date1,headlines,datamatrix]=ncfread2(filename);
        
datamatrix=dlmread(filename,'\t',1,2); 

%# read row headers

fid=fopen(filename,'r');
headerlines=fgets(fid);
index=findstr(headerlines,'}');
ncols=length(index); 
headlines={};
headlines(1)=headerlines(1:index(1));
for m=2:ncols
          headlines(m)=headerlines(index(m-1)+1:index(m));
endfor

%# read data as cell array of strings (need that for reading first two
columns as string)
in = fscanf(fid, '%c'); 
fclose(fid); 
 
lines = regexp(in, '(^|\n)[^\n]+', 'match'); %# split to lines
data = cellfun(@(in) regexp(in, '(\b|'')[^\t]+(''|\b)', 'match'), lines,
'UniformOutput', false); %# extract fields


Date1=cell2mat(cellfun(@(x) x(1),data,'UniformOutput',false)); 
Date1=char(Date1(1));

%# create time vector
zeit=(cellfun(@(x) x(2),data,'UniformOutput',false)); %# time as cell array
of strings

t=zeros(size(datamatrix,1),1);
timestring=char(zeit);
for jj=1:size(timestring,1)
    tstruct=strptime(timestring(jj,:),'%R');
          t(jj)=tstruct.hour+tstruct.min/60;
endfor

datamatrix=[t,datamatrix];
datamatrix(:,end)=[];


endfunction     


MarcelK wrote:
> 
> 
> I will now try to install Octave 3.4.3. and see if it works and if there's 
> an improvement in speed.
> 
> Marcel
> 


--
View this message in context: 
http://octave.1599824.n4.nabble.com/reading-text-file-with-textscan-annoyingly-slow-tp3972499p3985514.html
Sent from the Octave - General mailing list archive at Nabble.com.