Re: Importing large amounts of data

[Hannes]

> 
> Hm.  First, I had asked for a 2x2x3 example :)
> 
> Anyway, this takes three seconds on my pc, but that may depend on
> hardware (this is an AMD Phenom II X4 965):


Indeed it is 6 seconds here, had that confused.

> octave> tic;example;toc
> Elapsed time is 3.1 seconds.
> octave> whos
> Variables in the current scope:
> 
>    Attr Name          Size                     Bytes  Class
>    ==== ====          ====                     =====  ===== 
>         fam          19x19x55                 158840  double
>         families      1x144                     1152  double
>         indiv       144x144x55               9123840  double
> 
> Anyway, this is not the best way to load data into octave.  Much better
> is to write a file containing the data you need.  In text format for
> ease of debugging or in binary format for speed.  

I will read into the text format I will export that from my C++ program.


> That said, there is
> still no explanatin to the huge memory consumption you reported.  Try
> this:
> 
> clear all
> source <your file>
> whos
> 

I haven't yet been able to successfully run source(mybigfile) since it
has been going for a few days now at 8GB memory usage, so I quit it.

Thank you for your help. If it doesn't work with text format or still
consumes so much memory, I will be back ;)

Best,
Hannes

signature.asc
Description: OpenPGP digital signature