Re: memory limitations of octave

[Quentin Spencer]

Markus Sondermann wrote:

> Hello!
>
> I have tried to work with arrays of several millions of elements (say, 4
> millions). If I try to make a FFT, I get an error message 'memory exhausted'.
> I get the same message if I try to load binary data from files, e.g., if I try
> to load more than 11 million single precision floating point numbers. The
> problem occurs with the precompiled windows binary (version 2.1.50) as well as
> with the ovtave package coming along with Fedora Core 3. What can I do to
> handle large amounts of data? Compile from source with appropriate settings?
>  

I think this is a question of the hardware you're using as much as the 
software. I just did the following on my Fedora Core 4 box with Octave 
2.1.72:

a=randn(1,2^24);
b=fft(a);

The whos command revealed that this used a total of 400 MB of memory 
(note that the FFT is complex so b is twice as big as a). If you're 
loading 11 million single precision values, they are probably being 
converted to double in Octave, so you're using about 88 MB of memory, 
which should be no problem on Linux (I have no idea on Windows, but I 
think it has been discussed here before--try searching the archives). 
Note that if you have multiple variables of this size, you could fill up 
your physical memory pretty quickly, so you may want to see how much 
memory you're using before you try to load the large data file. If 
you're getting memory errors, it's probably because your computer 
doesn't have enough physical memory or swap space for what you're trying 
to do, unless you've exceeded the ~2 GB limitation of 32-bit addressing. 
The only way around that is to compile octave with --enable-64 flag on 
hardware that supports it.

-Quentin



-------------------------------------------------------------
Octave is freely available under the terms of the GNU GPL.

Octave's home on the web:  http://www.octave.org
How to fund new projects:  http://www.octave.org/funding.html
Subscription information:  http://www.octave.org/archive.html
-------------------------------------------------------------