[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: pinv() error message
From: |
siko1056 |
Subject: |
Re: pinv() error message |
Date: |
Wed, 11 Oct 2017 16:25:25 -0700 (MST) |
Ether Jones-2 wrote
> Please, what settings do I need to changeso Octave can complete the
> following computation without error. I have 3GB installed memory, and
> Process Explorer says I have over 2GB of physical memory available:
>
> + size (A)
> 25362 3331
>
> + size (b)
> 25362 1
>
> + X = pinv(A) * b;
>
> error: memory exhausted or requested size too large
> for range of Octave's index type
> error: execution failed
The Matrix itself is about 0.7 GB (= 25362 * 3331 * 8 Byte). Now explicitly
computing the pseudo-inverse is 2*0.7 GB (you see where the story goes),
plus some intermediate storage for the algorithm and your system is out of
memory (assuming, your operating system also requires some of the 3 GB). On
my machine (16 GB of main memory) I was able to figure out the memory
requirements using `pinv` for a random dense matrix and tortured my system:
+ A = rand (25362, 3331);
+ b = rand (25362, 1);
# 2.2 GB (Octave only) for intermediate computations
+ tic; X1 = pinv(A) * b; toc
Elapsed time is 452.245 seconds.
# 1.3 GB (Octave only) for intermediate computations
+ tic; X2 = A \ b; toc
Elapsed time is 31.0354 seconds.
# Both solutions almost identical
+ norm (X1 - X2)
ans = 1.0875e-14
+ max (X1 - X2)
ans = 6.1409e-16
Thus `pinv` failed for your example due to some missing megabytes I assume.
Anyway there is help for you:
Use Octave's swiss army knife solver "left division" [1], a.k.a "mldivide"
[2], a.k.a "backslash": `x = A \ b` like in the example above for `X2`.
In your case `A \ b` is equivalent to `pinv (A) * b`, but "mldivide" uses a
much more intelligent algorithm: it does not compute the pseudo-inverse
explicitly (you don't need that pseudo-inverse either!) and computes the
least-squares solution more directly, thus saving you lots of intermediate
memory and computation time.
Depending on the structure of A, you can further optimize your example:
1) Is your matrix dense, or are there more 0 entries, than non-zero ones?
==> Sparse matrices
2) Is your matrix structured somehow? ==> `A \ b` will find out, or
`matrix_type` [3]
HTH,
Kai
[1]: https://www.gnu.org/software/octave/doc/interpreter/Arithmetic-Ops.html
[2]: https://www.gnu.org/software/octave/doc/interpreter/XREFmldivide
[3]: https://www.gnu.org/software/octave/doc/interpreter/XREFmatrix_005ftype
--
Sent from: http://octave.1599824.n4.nabble.com/Octave-General-f1599825.html