[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: Loading ASCII data files quickly
From: |
Christoph Ellenberger |
Subject: |
Re: Loading ASCII data files quickly |
Date: |
Mon, 10 Aug 2009 17:03:29 +0200 |
I usually use sed to straighten the data and then do the load thing....
something like:
cmd=['sed -n -e "/^ *[0-9]/p" ' fname ">" Tfname];
if (system(cmd,1))
error("Fehler 1");
end
xx=load(Tfname);
where fname is the filename and tfname is some temporary filename which I
remove afterword with....
cmd=["rm " Tfname];
if (system(cmd,1))
error("Fehler 4");
end
but this depends on what structure do you want afterward. I sometimes have the
coulumnames in the file as well and create a cellarray with the datapoints
called Data.X, Data.Y etc. as well.
Hope this helps
Christoph
-------- Original-Nachricht --------
> Datum: Mon, 10 Aug 2009 16:24:07 +0200
> Von: Matthias Brennwald <address@hidden>
> An: address@hidden
> Betreff: Loading ASCII data files quickly
> Dear all
>
> I've got a few data files with a short header followed by two colums
> of data (see example below). The header data is marked by '*' signs,
> like this:
>
> ------------------------------
> * TMD data written by MATAA on 10-Aug-2009 15:37:39
> * MSW, 0 deg., no filter
> 0 -6.49888e-08
> 1.04167e-05 -1.77097e-07
> 2.08333e-05 2.53872e-07
> 3.125e-05 9.02216e-07
> 4.16667e-05 1.13457e-06
> 5.20833e-05 6.72286e-07
> 6.25e-05 -1.5686e-07
> 7.29167e-05 -6.71451e-07
> 8.33333e-05 -5.46739e-07
> 9.375e-05 -7.39792e-08
> 0.000104167 1.52549e-07
> 0.000114583 -9.41797e-08
> .
> .
> .
> ------------------------------
>
> I tried to read line by line using fgetl(...) into a string. If the
> line starts with an '*', it is treated as header line. Otherwise it is
> transformed into numbers using str2num(...). This line-by-line
> procedure works, but is too slow for large files. I believe it would
> be much more efficient to read only the header lines using this
> method, and then use a file command that loads the rest of the data in
> one step. Any hints or ideas how I can do this?
>
> Matthias
>
>
> ----
> Matthias Brennwald, Käferholzstrasse 173, CH-8046 Zürich, +41 44 364
> 17 03
>
>
> _______________________________________________
> Help-octave mailing list
> address@hidden
> https://www-old.cae.wisc.edu/mailman/listinfo/help-octave
--
GRATIS für alle GMX-Mitglieder: Die maxdome Movie-FLAT!
Jetzt freischalten unter http://portal.gmx.net/de/go/maxdome01
- Loading ASCII data files quickly, Matthias Brennwald, 2009/08/10
- Re: Loading ASCII data files quickly, Torsten Lange, 2009/08/10
- Re: Loading ASCII data files quickly, Francesco Potorti`, 2009/08/10
- Re: Loading ASCII data files quickly,
Christoph Ellenberger <=
- Re: Loading ASCII data files quickly, Matthias Brennwald, 2009/08/10
- Re: Loading ASCII data files quickly, Christoph Ellenberger, 2009/08/10
- Message not available
- Re: Loading ASCII data files quickly, Matthias Brennwald, 2009/08/10
- Re: Loading ASCII data files quickly, Prof M. N. Anandaram, 2009/08/13
- Re: Loading ASCII data files quickly, Jaroslav Hajek, 2009/08/13
Re: Loading ASCII data files quickly, Sergei Steshenko, 2009/08/11