Re: reading data from ascii files.

[Veloci]

Ok, after some trial and error, i've got a working version. 

As i wrote the dlmread function is realy slow if you try to read big
packages. So i changed the whole thing into a read piece by piece:

function
[val_max,cnt_gr_as,val_min,cnt_lw_as,val_mean,val_std,val_count,data] =
read_cal_data_oct(filename,gr_as,lw_as)

nr_of_header_rows = 9;
nr_of_rows2read = 1000;

fid = fopen(filename);

rnum = 0;
while (feof(fid)~= 1)
  rnum = rnum + 1;
  l = fgetl (fid);
end;

%allocating memory
data =zeros((rnum-1)-(nr_of_header_rows-1),4);  %rnum-1 because last row in
my files is empty
disp(size(data));

%delimited read of the data

%calculate number of fitting range vectors 
%range vector = [first_row,first_column,last_row,last_column]

nr_rv_cnt = floor((rnum-1-nr_of_header_rows)/nr_of_rows2read);
disp(nr_rv_cnt );

for j = nr_of_header_rows:nr_of_rows2read:((nr_rv_cnt-1)*nr_of_rows2read+9)
                r = [j 1 (j+nr_of_rows2read-1) 4];
                disp(r);
                disp(j);
                be= j- (nr_of_header_rows-1);
                en= be+ nr_of_rows2read-1;
                disp(be);
                disp(en);
                data(be:en,:) = dlmread(filename,'\t',r);
end;

%calculate last range vectors

j= j+nr_of_rows2read;
r = [j 1 (rnum-1) 4];
disp(r);

be= en + 1;
en= be +  (rnum-1-j);
disp(be);
disp(en);
data(be:en,:) = dlmread(filename,'\t',r);

fclose(fid);

%data processing
...


With these changes i can process my files in a matter of some
seconds/minutes (the bigger "nr_of_rows2read", the slower it gets).  
Maybe with some adjustment it can be helpful to some of you. 

Greetings,
Veloci.

PS: The disp() calls are not needed. They were just added for debugging
reasons.
-- 
View this message in context: 
http://octave.1599824.n4.nabble.com/reading-data-from-ascii-files-tp2291167p2292026.html
Sent from the Octave - General mailing list archive at Nabble.com.