Re: pinv() error message

[siko1056]

Ether Jones-2 wrote
> Please, what settings do I need to changeso Octave can complete the
> following computation without error. I have 3GB installed memory, and
> Process Explorer says I have over 2GB of physical memory available:
> 
> + size (A)
>    25362    3331
> 
> + size (b)
>    25362    1
> 
> + X = pinv(A) * b;
> 
> error: memory exhausted or requested size too large
> for range of Octave's index type
> error: execution failed

The Matrix itself is about 0.7 GB (= 25362 * 3331 * 8 Byte).  Now explicitly
computing the pseudo-inverse is 2*0.7 GB (you see where the story goes),
plus some intermediate storage for the algorithm and your system is out of
memory (assuming, your operating system also requires some of the 3 GB).  On
my machine (16 GB of main memory) I was able to figure out the memory
requirements using `pinv` for a random dense matrix and tortured my system:

+ A = rand (25362, 3331);
+ b = rand (25362, 1);

# 2.2 GB (Octave only) for intermediate computations
+ tic; X1 = pinv(A) * b; toc
Elapsed time is 452.245 seconds.

# 1.3 GB (Octave only) for intermediate computations
+ tic; X2 = A \ b; toc
Elapsed time is 31.0354 seconds.

# Both solutions almost identical
+ norm (X1 - X2)
ans =    1.0875e-14
+ max (X1 - X2)
ans =    6.1409e-16

Thus `pinv` failed for your example due to some missing megabytes I assume.
Anyway there is help for you:

Use Octave's swiss army knife solver "left division" [1], a.k.a "mldivide"
[2], a.k.a "backslash": `x = A \ b` like in the example above for `X2`.

In your case `A \ b` is equivalent to `pinv (A) * b`, but "mldivide" uses a
much more intelligent algorithm: it does not compute the pseudo-inverse
explicitly (you don't need that pseudo-inverse either!) and computes the
least-squares solution more directly, thus saving you lots of intermediate
memory and computation time.

Depending on the structure of A, you can further optimize your example:

1) Is your matrix dense, or are there more 0 entries, than non-zero ones?
==> Sparse matrices
2) Is your matrix structured somehow? ==>  `A \ b` will find out, or
`matrix_type` [3]

HTH,
Kai

[1]: https://www.gnu.org/software/octave/doc/interpreter/Arithmetic-Ops.html
[2]: https://www.gnu.org/software/octave/doc/interpreter/XREFmldivide
[3]: https://www.gnu.org/software/octave/doc/interpreter/XREFmatrix_005ftype



--
Sent from: http://octave.1599824.n4.nabble.com/Octave-General-f1599825.html