Re: reading data from ascii files.

[Siep Kroonenberg]

On Fri, Jul 16, 2010 at 02:58:43AM -0700, Veloci wrote:
> 
> Hi everyone,
> 
> I know that the topic sounds familiar to many of you, so i apologize for any
> annoyance it may cause.
> My problem with the data input is not so much the "what functions to use",
> more like "making it faster". 
> Starting point are a couple of data files in ascii format with a specified
> number of header rows (data on the experiment and so on) followed by a
> unspecified number of data-rows (depending on the measurement time and the
> sampling rate i use). 
> What i want to do: I want to implement a automated input of these data files
> into octave matrices and calculate data from these (as the most people
> would).
> What i want to use for that: The most promising function seems to be
> "dlmread", as my data is organised in rows with a fixed number of columns
> and a fixed delimiter.
> My problem: As the number of rows is not specified, i can't give dlmread a
> range vector with the coordinates of the last row. This is not a tragic
> problem for dlmread, because it still works with only the starting row and
> column, but it is a rather time consuming matter concerning the huge data it
> has to read. It would be much faster if i could allocate the needed memory
> in a zero matrix with the size of the data. For this i need to know the
> number of rows in my ascii file. You could say, "Open the file in a good
> editor and seek for the end of rows". That would be a solution for manual
> data processing, but not for a automated procedure. 
> The solution i seek: I've searched on the net for a solution to my problem,
> but i couldn't find anything that would be possible in Octave. In MATLAB
> there was a possibility with memmapfile, that calculates the number of rows
> in a file. But memmapfile is not implemented in Octave. So i tried to find
> something similar, but couldn't find any example. I could imagine using the
> "eof"-pointer, but i don't know how to calculate the end row from that
> position.  
> 
> If there is any solution that you could suggest, it would be much
> appreciated.

Couldn't you just read and count all lines without parsing?

rnum = 0;
while (! feof(fid))
  rnum = rnum + 1;
  l = fgetl (fid);
end

-- 
Siep Kroonenberg