Re: Problem with large array

[Nicholas Jankowski]

first thing that jumps out is that Octave doesn't know how big X_big
and Y_big will be, so memory handling for them will be poor. Here's a
Matlab help file describing preallocation:
http://www.mathworks.com/support/solutions/en/data/1-18150/

Seems like you should know how big it will wind up being, you can
create a zero-filled matrix of the right size prior to entering the
loop. that may help.

How quickly does it execute the print command relative to the
computation time? any text output to screen will slow down a fast
loop.

nickj

On Tue, Mar 20, 2012 at 10:18 AM, George <address@hidden> wrote:
> Hi,
>
> I am creating a large matrix, and got the following error:
>
> error: memory exhausted or requested size too large for range of Octave's
> index type -- trying to return to prompt.
>
> The size of the matrix when the error occurs is 23,289 x 2,387 or about
> 55,000,000 elements.
>
> Working in WIndows XP, on Octave 3.2.4. I have a 64 bit CPU, but I think
> running Windows 32 bit version.
>
> I am combining the elements of two matrices. Here is the code:
>
>   X_big = [];
>   Y_big = [];
>   nm = size(R, 1);
>   np = size(R, 2);
>   for m_counter = 1 : nm
>     printf("%i...", m_counter);
>     for p_counter = 1 : np
>       if R(m_counter, p_counter) == 1
>         this_row = [features(m_counter, :), Y'(p_counter,:) ];
>         X_big = [X_big; this_row];
>         Y_big = [ Y_big; Y(m_counter, p_counter) ];
>       endif
>     endfor
>   endfor
>
> The final matrix should be approximately of size 34,000 x 2,387 or about
> 80,000,000 elements.
>
> Can you please answer a few questions?
>
> 1. How can I fix this error?
> 2. This process is very slow...taking about 6 hours (if it finished.) Is
> there a better way to do this?
>
> Thank you for your help.
>
> George
>
>
> _______________________________________________
> Help-octave mailing list
> address@hidden
> https://mailman.cae.wisc.edu/listinfo/help-octave
>