help-octave
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Reading big files without Memory exhausted message?


From: Mariano Wis
Subject: Reading big files without Memory exhausted message?
Date: Wed, 14 Apr 2004 00:41:51 +0200

Hi all,
 
    The summary of this message is:
 
    Can  Octave be configured in any way to read big files and write them in big arrays without the "memory exhaust -- trying to return propmpt" message and the fail of the program?
 
    I'm trying to read a big file that is organized in samples of 7 doubles each. I have used two different strategies to read this file (see sample codes below). When I try to read a file that is not too big (about 150 Mbytes)  I have no problem to read it. The problems arises when I try to read bigger files (for example 230 Mbytes). At a certain point, I began to read the message: "memory exhaust -- trying to return propmpt" and the program window is closed. This happens at differents points of the sample codes. With sample code 1 the message is shown 2 times after the last fread. With sample code 2 the message is shown 7 times after the first fread. In both cases the Octave program is aborted and the window is closed.
 
    I'm using octave-forge-windows 2.1.50 windows installation on a 1 Gb RAM / Athlon XP 2400+ system running Windows XP and on a 512 Mb RAM / Pentium4 system running Windows 2000 and I have the same effects on both systems.
 
My questions are:
 
- Is the problem related with the configuration of Octave/Cygwin?
- If so, can this configuration be changed to be able to read and store in memory such big files?
 
Thanks beforehand,
 
Mariano Wis
 
 
 
## *** SAMPLE CODE 1 ***
imu_file = 'c:\\tstsds\\input_files\\imu_20030701.dat';
n_fields = 7; ## Number of fields for each acquisition
data_size = 8; ## Size of each data (8 is double)
 
f_info = stat(imu_file);
f_size = f_info.size / (n_fields * data_size); ## Number of samples
 
f_id = fopen(imu_file,'rb','native');
 
time = fread (f_id,f_size,'double',48,'native');
 
fseek (f_id,8,SEEK_SET); ## Locate to read wx
wx = fread (f_id,f_size,'double',48,'native');
 
fseek (f_id,16,SEEK_SET); ## Locate to read wy
wy = fread (f_id,f_size,'double',48,'native');
 
fseek (f_id,24,SEEK_SET); ## Locate to read wz
wz = fread (f_id,f_size,'double',48,'native');
 
fseek (f_id,32,SEEK_SET); ## Locate to read ax
ax = fread (f_id,f_size,'double',48,'native');
 
fseek (f_id,40,SEEK_SET); ## Locate to read ay
ay = fread (f_id,f_size,'double',48,'native');
 
fseek (f_id,48,SEEK_SET); ## Locate to read az
az = fread (f_id,f_size,'double',48,'native');
 
fclose(f_id);
 
 
## *** SAMPLE CODE 2 ***
imu_file = 'c:\\tstsds\\input_files\\imu_20030701.dat';
n_fields = 7; ## Number of fields for each capture
data_size = 8; ## Size of each data
f_info = stat(imu_file);
f_size = f_info.size / (n_fields * data_size); ## Number of samples
 
f_id = fopen(imu_file,'rb','native');
imu_vec = fread (f_id,[n_fields,f_size],'double',0,'native');
 
time = imu_vec(1,1:f_size);
wx = imu_vec(2,1:f_size);
wy = imu_vec(3,1:f_size);
wz = imu_vec(4,1:f_size);
ax = imu_vec(5,1:f_size);
ay = imu_vec(6,1:f_size);
az = imu_vec(7,1:f_size);
 
clear imu_vec;
 
fclose(f_id);
 
 
 

---
Outgoing mail is certified Virus Free.
Checked by AVG anti-virus system (http://www.grisoft.com).
Version: 6.0.655 / Virus Database: 420 - Release Date: 09.04.2004

reply via email to

[Prev in Thread] Current Thread [Next in Thread]