[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
cannot load large matrix, new datatype/fseek
From: |
daniel heiserer |
Subject: |
cannot load large matrix, new datatype/fseek |
Date: |
Fri, 18 Jun 2004 02:58:31 -0500 |
Hi,
I have a large matrix in the following mat file:
8153243903 14. Jun 16:49 ftn51.mat
The matrix is approximately 5,000,000 x 200 and needs around 8GB of
space.
Unfortunately octave cannot read the matrix. It either stops immediatley
on my 32bit machine (of course) or on a 8GB 64bit machine it simply
crashes.
I only need a few rows of that matrix, but all columns.
I have two attesmpts for reading that matrix:
1) write an m-file which reads only the elements I want.
For that I am not sure if I can "naviagte" using fseek in a file larger
then 2GB.
If anyone up to know has written an m-file based .mat parser I would be
glad for
any help
2) Implement a new data type OOC-matrix. This datatype holds only the
few values
describing the matrix in memory (imaginary flag, n-columns,n-rows).
The rest of the matrix is a blob on disk.
Whenever this datatype is used in a function or whatever it shall load
the necessary elements from disk.
As I haver never implemented a new datatype I ask for some documentation
or examples so I can start realizing it. Also I must somehow be able to
tell "load" that the matrix in the file shall be assigned to that
datatype.
I am glad for any help,
daniel
-------------------------------------------------------------
Octave is freely available under the terms of the GNU GPL.
Octave's home on the web: http://www.octave.org
How to fund new projects: http://www.octave.org/funding.html
Subscription information: http://www.octave.org/archive.html
-------------------------------------------------------------
- cannot load large matrix, new datatype/fseek,
daniel heiserer <=