Re: out of memory or dimension too large for Octave's index type

[Nicholas Jankowski]

On Tue, Jun 17, 2014 at 10:22 AM, dkeck <address@hidden> wrote:
> Octave 3.8.0 32-bit.
> 4GB RAM of which 3,25 are visible to the OS (WinXP 32bit) of which are 60%
> in use before building A. So 1.3 GB of free space for A.
>
> There are only temporary variables in the workspace visible while building
> A.
> A_final=[A_1;A_2;...] where A_INT are preallocated with zeros but A_final is
> not preallocated but simply build as shown here.
> What does "are you forcing duplicate copies" mean in this respect?
>
> Yes, 'A' should be a double precision array since I did not modify any A_INT
> while preallocating.

maybe not the correct terminology, but by duplicate copies I mean that
certain methods of building an array can require Octave to hold two
copies of the array at the same time. This I believe is usually the
case when you force a resizing of an array. Someone else can chime in
if I butcher this but something like:
----------
a = [1 2 3];
A=[];
for i= 1:5
  A = [A;a]
endfor
-----------

each time you append onto A, I believe Octave has to hold both the old
and new A in memory to complete the operation. Hence, for large arrays
with small additions, you effectively double the amount of memory
required for the operation. Not 100% sure that's your problem here,
but it comes up often when people are manipulating large arrays.

Checking: if you're using double precision, each element is 8 bytes,
and you have 5280 x 4608 = 24,330,240 elements, that's only
194,641,920 bytes. So, after all of that, unless you're doing
something to keep 7 of those in memory simultaneously, or have
something else sitting there occupying space, that alone shouldn't be
the problem.

sharing the relevant code, especially if you can simplify to the
error, may allow someone to spot the problem.