[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: Octave Limits
From: |
Daryl Warkentin |
Subject: |
Re: Octave Limits |
Date: |
Mon, 24 Nov 2003 10:42:54 -0600 |
On Mon, 2003-11-24 at 09:54, Przemek Klosowski wrote:
> Upon further inspection I have found that Octave does seem to have a
> file size limit of 2GB.
> ....
> I got the following error while running one of my scripts:
>
> File size limit exceeded
>
> And the file was exactly 2GB. This happens on both ext3 and FAT32
> filesystems. I am running a 2.4.21 kernel.
>
> Ah, but that is a separate issue: currently, octave internally can only
> handle a 2GB single object, because it uses int pointers. That is why
> I said that there should be no limit if octave makes individual, small reads
> and writes to a file---what you are doing is probably reading into
> a single array.
Nope. I store approximately 500MB in memory at a time.
octave:1> testfile = fopen('/home/warkentin/Stuff/testdata', 'w+', 'native');
octave:2> testdata = ones(50000000, 1);
octave:3> while(1)
> fwrite(testfile, testdata, 'char', 0, 'native');
> end
panic: File size limit exceeded -- stopping myself...
attempting to save variables to `octave-core'...
address@hidden Stuff]$ ls -l testdata
-rw-rw-r-- 1 warkentin warkentin 2147483647 Nov 24 10:39 testdata
> BTW, you will be in a world of pain if you are trying to manipulate
> 2GB+ memory objects on a 32-bit architecture such as x86. Octave may
> need a fix, but you do need to run octave on a platform such as MIPS,
> SPARC, Itanium, or AMD x86-64. Since MIPS, SPARC, Itanium are either
> outrageously expensive, or low performance, or both, AMD Opteron class
> 64-bit CPUs are basically your only choice.
Stuck with an AMD 2800+ and 1.5GB RAM. Would love an Opteron.
> DISCLAIMER: this is my personal opinion, not known to, or reflecting
> the position of, my employer; contents may settle; void where
> prohibited, etc.
>
>
> p
--
Daryl Warkentin, P. Eng.
Hardware Engineering
SED Systems
www.sedsystems.ca
-------------------------------------------------------------
Octave is freely available under the terms of the GNU GPL.
Octave's home on the web: http://www.octave.org
How to fund new projects: http://www.octave.org/funding.html
Subscription information: http://www.octave.org/archive.html
-------------------------------------------------------------
- Octave Limits, (continued)
- Octave Limits, John W. Eaton, 2003/11/21
- Re: Octave Limits, Przemek Klosowski, 2003/11/21
- Re: Octave Limits, Daryl Warkentin, 2003/11/21
- Re: Octave Limits, Przemek Klosowski, 2003/11/21
- Re: Octave Limits, Daryl Warkentin, 2003/11/21
- Re: Octave Limits, John W. Eaton, 2003/11/21
- Re: Octave Limits, Dmitri A. Sergatskov, 2003/11/21
- Re: Octave Limits, Przemek Klosowski, 2003/11/24
- Re: Octave Limits,
Daryl Warkentin <=
- Re: Octave Limits, Przemek Klosowski, 2003/11/21