Re: Loading ASCII data files quickly

[Sergei Steshenko]

--- On Mon, 8/10/09, Matthias Brennwald <address@hidden> wrote:

> From: Matthias Brennwald <address@hidden>
> Subject: Loading ASCII data files quickly
> To: address@hidden
> Date: Monday, August 10, 2009, 7:24 AM
> Dear all
> 
> I've got a few data files with a short header followed by
> two colums  
> of data (see example below). The header data is marked by
> '*' signs,  
> like this:
> 
> ------------------------------
> * TMD data written by MATAA on 10-Aug-2009 15:37:39
> * MSW, 0 deg., no filter
> 0    -6.49888e-08
> 1.04167e-05    -1.77097e-07
> 2.08333e-05    2.53872e-07
> 3.125e-05    9.02216e-07
> 4.16667e-05    1.13457e-06
> 5.20833e-05    6.72286e-07
> 6.25e-05    -1.5686e-07
> 7.29167e-05    -6.71451e-07
> 8.33333e-05    -5.46739e-07
> 9.375e-05    -7.39792e-08
> 0.000104167    1.52549e-07
> 0.000114583    -9.41797e-08
> .
> .
> .
> ------------------------------
> 
> I tried to read line by line using fgetl(...) into a
> string. If the  
> line starts with an '*', it is treated as header line.
> Otherwise it is  
> transformed into numbers using str2num(...). This
> line-by-line  
> procedure works, but is too slow for large files. I believe
> it would  
> be much more efficient to read only the header lines using
> this  
> method, and then use a file command that loads the rest of
> the data in  
> one step. Any hints or ideas how I can do this?
> 
> Matthias
> 
> 
> ----
> Matthias Brennwald, Käferholzstrasse 173, CH-8046 Zürich,
> +41 44 364  
> 17 03
> 
> 


There is Perl for many systems, including Windows. And 'grep'.

For a simple case like this 'grep' is sufficient to get rid of header.

Regards,
  Sergei.