Re: reading data using fgets in while loop

[indium]

On Mon, Mar 12, 2012 at 12:13:50PM -0400, financial engineer wrote:
> 
> 
> 
> > Date: Mon, 12 Mar 2012 15:48:40 +0000
> > From: address@hidden
> > To: address@hidden
> > CC: address@hidden
> > Subject: Re: reading data using fgets in while loop
> > 
> > > > Date: Mon, 12 Mar 2012 11:37:37 +0000
> > > > From: address@hidden
> > > > To: address@hidden
> > > > CC: address@hidden
> > > > Subject: Re: reading data using fgets in while loop
> > > > 
> > > > On Sun, Mar 11, 2012 at 04:00:36PM -0700, newbie_octave wrote:
> > > > > I have a .csv file with the following data
> > > > > 
> > > > > 01/03/2012,H (Mar 12),26.750000,2584
> > > > > 01/04/2012,H (Mar 12),26.280000,2330
> > > > > 01/05/2012,H (Mar 12),26.000000,3198
> > > > > 01/06/2012,H (Mar 12),25.500000,3045
> > > > > 01/09/2012,H (Mar 12),25.350000,2314
> > > > > 01/10/2012,H (Mar 12),25.050000,2685
> > > > > 
> > > > > and I am a newbie to octave. I ran the following command to read the 
> > > > > above
> > > > > data into octave
> > > > > 
> > > > > X=csvread("/tmp/hist.csv")
> > > > > 
> > > > > but it returns the following matrix.
> > > > > 
> > > > > 1.0000e+00   0.0000e+00   2.6750e+01   2.5840e+03
> > > > >    1.0000e+00   0.0000e+00   2.6280e+01   2.3300e+03
> > > > >    1.0000e+00   0.0000e+00   2.6000e+01   3.1980e+03
> > > > >    1.0000e+00   0.0000e+00   2.5500e+01   3.0450e+03
> > > > >    1.0000e+00   0.0000e+00   2.5350e+01   2.3140e+03
> > > > >   1.0000e+00   0.0000e+00   2.5050e+01   2.6850e+03
> > > > > 
> > > > > 
> > > > > Obviously, it is not reading the text correctly
> > > > > so I am now using fgets() as follows:
> > > > > 
> > > > > fid=fopen(fname);
> > > > > fout="out.mat"
> > > > > global tline;
> > > > > global tempstr;
> > > > > while 1
> > > > > tline = fgets(fid);
> > > > > sep=",";
> > > > > tempstr=strsplit(tline, sep);
> > > > > dt=tempstr(1);
> > > > > cname=tempstr(2);
> > > > > price=str2double(tempstr(3));
> > > > > volume=str2double(tempstr(4));
> > > > > if ~ischar(tline), break,end;
> > > > > end
> > > > > disp(tline);
> > > > > fclose(fid);
> > > > > 
> > > > > but when I run the script, it returns -1 due to disp(tline) being 
> > > > > after the
> > > > > end. I want to be able to compute the volume-weighted average price, 
> > > > > and not
> > > > > have to do it all within the script that reads the data from the 
> > > > > file. But,
> > > > > given that I cannot access any of the variables outside the while 
> > > > > loop, I am
> > > > > stuck. Ideally, I could have used csvread() or dlmread() but that is 
> > > > > messing
> > > > > up the string fields in my file. Can anyone please suggest how I fix 
> > > > > this so
> > > > > I can manipulate the variables outside the while loop that gets the 
> > > > > data
> > > > > from the .csv file. thanks!
> > > > > 
> > > > > 
> > > > > 
> > > > > --
> > > > > View this message in context: 
> > > > > http://octave.1599824.n4.nabble.com/reading-data-using-fgets-in-while-loop-tp4464844p4464844.html
> > > > > Sent from the Octave - General mailing list archive at Nabble.com.
> > > > 
> > > > Hi, 
> > > > I would try while(!eof(fid)) instead of while(1). 
> > > > This would stop the while loop when the whole file is read. 
> > > > 
> > > > as an example, I've copied a part of a script that I use myself. This 
> > > > also does some date conversion to 'seconds since epoch'. 
> > > > To read the manual, do 'doc eof' or 'doc mktime', etc.
> > > > 
> > > > ------------------------------------------------
> > > > params27C.date:
> > > > 05/12/11 18:56:37 0.001728386 16.43405 1.323835
> > > > 05/12/11 19:26:39 0.001676934 16.41423 1.334429
> > > > 05/12/11 19:56:40 0.001599179 16.40604 1.351362
> > > > 05/12/11 20:54:10 0.001547091 16.35170 1.361025
> > > > ------------------------------------------------
> > > > 
> > > > script:
> > > > 
> > > > pos27f=fopen('./fits2012/27C/params27C.date');
> > > > n=0;
> > > > while (!feof(pos27f))
> > > >         n++
> > > >         line=strsplit(fgetl(pos27f),' ')
> > > >         datestring=sprintf('%s %s',line{1},line{2});
> > > >         pos27sec(n)=mktime(strptime(datestring,'%d/%m/%y %T'));
> > > >         pos27(n)=str2num(line{4});
> > > > endwhile
> > > > fclose(pos27f)
> > > > 
> > > > 
> > >                                     
> > 
> > On Mon, Mar 12, 2012 at 11:23:56AM -0400, financial engineer wrote:
> > > 
> > > 
> > > Thanks for the script...I rewrote mine as follows:
> > > fname="/tmp/VIXts.csv"
> > > fid = fopen(fname);
> > > n=0;
> > > while (!feof(fid))
> > > n++
> > > tline=strsplit(fgetl(fid),',')
> > > datestring=sprintf('%s',tline{1});
> > > fid(n)=str2num(tline{3});
> > > endwhile
> > > fclose(fid)
> > > 
> > > but when it encounters endwhile, it gives the following error:
> > > > endwhile
> > > error: feof: invalid stream number = -1
> > > 
> > > 
> > 
> > Dear financial engineer,
> > 
> > please put your 'reply-text' at the bottom of the thread (like I do here), 
> > so that the thread is readable from top-to-bottom.
> > 
> > feof() complains because the fid is not a positive number. The fid is not 
> > positive, which most certainly means that the file
> > was never opened. For instance, the file doesn't exist, the name or 
> > directory is spelled wrong, the file permission on the filesystem is
> > wrong, etc, etc. 
> > 
> > Can you try to do it manually, step by step, by starting typing the 
> > commands on the octave command-line?
> > Try other files, skip the while() loop in first instance, check the file 
> > name, try fid=fopen('/tmp/VIXts.csv') directly, etc, etc.
> 
> hi Indium,
> In fact the file was missing, so now it is capturing the price and volume, 
> but I only get one date. To fix that I tried, using datestring(n) as below:
> 
> here is my updated code:
> fname="/tmp/VIXts.csv"
> fid = fopen(fname);
> n=0;
> while (!feof(fid))
> n++
> tline=strsplit(fgetl(fid),',')
> datestring(n)=sprintf('%s',tline{1});
> %cname(n)=sprintf('%s',tline{2});
> price(n)=str2double(tline{3});
> volume(n)=str2num(tline{4});
> endwhile
> fclose(fid)
> 
> Now when I run it, I get the following error due to 
> datestring(n)=sprintf('%s',tline{1});
> fname = /tmp/VIXts.csv
> ans = 0
> tline =
> 
> {
>   [1,1] = 01/03/2012
>   [1,2] = H (Mar 12)
>   [1,3] = 26.750000
>   [1,4] = 2584
> }
> 
> error: A(I) = X: X must have the same size as I
> 
> how do I capture all the datestrings.
> 
> thanks,
> Bobby
> 
>                                         

Hi Bobby,

sprintf will give you a string, but you index datestring as a matrix.

octave:1> e(1)="werwerwe"
error: A(I) = X: X must have the same size as I
octave:1> e{1}="werwerwe"
e = 
{
  [1,1] = werwerwe
}
octave:2> e{2}="rrrrrrr"
e = 
{
  [1,1] = werwerwe
  [1,2] = rrrrrrr
}
octave:3> 

So, you can store your strings with {} instead of ().

Another possibility is converting the date to a integer, as I have shown in my 
script. Then you can store those in a matrix/vector using the () index.

Be aware with these cells: if you generate them with {}, you can retrieve 
elements back with {}, but also with (). I don't always grasp the idea behind 
that myself...