Re: question on reading ascii data from a file.

[Ben Abbott]

On Oct 25, 2012, at 7:16 AM, address@hidden wrote:

> Dear listmembers,
> I have a (probably ...) simple question. When reading data from a file (say, 
> two columns of numbers), the fastest way to access the data to do a
> 
> infile=fopen("name", "r");
> data=fscanf(infile, "%f %f");
> fclose(infile);
> 
> however, doing this you cannot cope with lines having a different shape and / 
> or being comments. To circumvent this problem, I started reading line by 
> line, removing whitespace from the front of the line and check wheter the 
> first non whitespace  character of the line is a %, a ;, a # or a similar 
> character. This actually works, however, it is very slow.
> 
> Does someone of you probably have an idea how to handle this (improve speed)?
> 

Vectorizing your approach should help.  The code below is untested.

# Read the file's contents
text = fileread (filename);
# Kludgy way to split on EOL
text = strrep (text, [char(13), char(10)], char(10));
text = strrep (text, char(13), char(10));
text = strsplit (text, char(10));
# Remove empty lines
fun = @(x) isempty (x);
n = cellfun (fun, text, 'uniformoutput', true);
text(n) = [];
# Remove comment lines
comment = {"%", "#", ";"};
for c = 1:numel(in.comment)
  fun = @(x) strncmp (x, in.comment{c}, 1);
  n = cellfun (fun, text, 'uniformoutput', true);
  text(n) = [];
end
# Convert text to numbers
foobar = @(str) sscanf (str, "%f");
cells = cellfun (foobar, a, "UniformOutput", false)
nums = cell2mat (cells).';

Ben