[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
File size limit?
John W. Eaton
File size limit?
Thu, 15 Apr 1999 21:46:18 -0500 (CDT)
On 16-Apr-1999, Andre Bonfrer <address@hidden> wrote:
| Does Octave have a limit imposed on reading in files from ascii
As far as I know, you are limited by the amount of memory you have, or
the largest value that can be represented in a signed int, which
Octave uses internally for storing matrix dimensions (I know, it
should probably be size_t, but it is not, at least yet).
| I was trying to use fscanf to read in a file with about 4million rows, 7
| columns (fixed length):
| 294 2 298 11 2.6200 1 1
| 294 2 299 9 2.5100 1 1
| 294 2 300 6 2.4900 1 1
| 294 2 301 5 2.9900 1 1
| 294 2 302 1 2.4900 1 1
| Using fscanf:
| infid = fopen("data.dat","r","native");
I think you probably want to write
| Each time the size of wb is limited s.t. count=33795
| Can anyone help me? Is there some parameter I have to set in
| configuration to allow Octave to recognise beyound 33795?
Do you see any error messages? What version of Octave are you using?
I had no trouble reading a file like yours containing several hundred
thousand lines. I ran out of memory trying to read a file with 4
million lines (That's not surprising, since a 7x4000000 matrix
requires more than 210 MB. If you don't tell Octave how large the
matrix is in the fscanf call, it has to reallocate a buffer many
times, starting with a result matrix containing just nr*32 elements,
then doubling the number of columns each time it runs out of room.
If you want to optimize this for larger matrices, you can. Just take
a look at the code the code that handles scanf conversions in