Aw: Re: Re: memory leak in regexprep ? (windows/3.7.2+/VS)

[Christoph Ellenberger]

Ok thanks for testing, I filed a bug (#38616)
christoph

> Gesendet: Donnerstag, 28. März 2013 um 04:05 Uhr
> Von: "Sergei Steshenko" <address@hidden>
> An: "address@hidden" <address@hidden>, "Christoph Ellenberger" 
> <address@hidden>
> Cc: "address@hidden" <address@hidden>
> Betreff: Re: Aw: Re: memory leak in regexprep ? (windows/3.7.2+/VS)
>
> 
> 
> 
> 
> ----- Original Message -----
> > From: Tatsuro MATSUOKA <address@hidden>
> > To: Christoph Ellenberger <address@hidden>
> > Cc: address@hidden
> > Sent: Thursday, March 28, 2013 4:07 AM
> > Subject: Re: Aw: Re: memory leak in regexprep ? (windows/3.7.2+/VS)
> > 
> > --- On Wed, 2013/3/27, Christoph Ellenberger wrote:
> >>  > --- On Wed, 2013/3/27, Tatsuro MATSUOKA wrote:
> >>  > > --- On Wed, 2013/3/27, Tatsuro MATSUOKA wrote:
> >>  > > > --- On Tue, 2013/3/26, Christoph Ellenberger  wrote:
> >>  > > > > I am experiencing some strange behavior and wanted to 
> > check if it is  a bug or me....
> >>  > > > > It happens when I try to read in and manipulating some 
> > ascii data files. It only happens with the VS version of octave. (tested 
> > with 
> > several gcc versions on windows and didn't showed up).
> >>  > > > > Ok here is some example code to reproduce:
> >>  > > > > First generating an ascii file:
> >>  > > > > s="0123456789";
> >>  > > > > for i=1:21, s=cstrcat(s,s);,endfor
> >>  > > > > save TestData.dat s
> >>  > > > >  
> >>  > > > > Then readin and manipulate:
> >>  > > > > fid = fopen("TestData.dat","r");
> >>  > > > > s = fscanf(fid,'%c');
> >>  > > > > fclose(fid);
> >>  > > > > s = regexprep(s,'0','2');
> >>  > > > >  
> >>  > > > > Clear all and repeat:
> >>  > > > > clear all
> >>  > > > > 
> >>  > > > > fid = fopen("TestData.dat","r");
> >>  > > > > s = fscanf(fid,'%c');
> >>  > > > > fclose(fid);
> >>  > > > > s = regexprep(s,'0','2');
> >>  > > > >  
> >>  > > > > Depending on datasize and how many replacement were 
> > performed it either fails on the new readin or regexprep with a: "error: 
> > out of memory or dimension too large for Octave's index type"
> >>  > > > > Any help is appreciated
> >>  > > > > Christoph
> >>  > > > 
> >>  > > > I have tested on octave-3.6.2 VS, 3.6.2 MinGW, 3.7.2+ VS and 
> > 3.6.2 cygwin
> >>  > > > (win 7 64 bit Home premium, 4GB memory)
> >>  > > > 
> >>  > > > 
> >>  > > > 1. octave-3.6.2 VS and 3.6.2 
> > MinGW*****************************
> >>  > > > >> fid = 
> > fopen("TestData.dat","r");
> >>  > > > >> s = fscanf(fid,'%c');
> >>  > > > warning: range error for conversion to character value
> >>  > > > >> fclose(fid);
> >>  > > > >> s = regexprep(s,'0','2');
> >>  > > > >> clear all
> >>  > > > >> fid = 
> > fopen("TestData.dat","r");
> >>  > > > >> s = fscanf(fid,'%c');
> >>  > > > error: memory exhausted or requested size too large for 
> > range of Octave's index type -- trying to return to prompt
> >>  > > > 
> > ********************************************************************
> >>  > > > 
> >>  > > > 
> >>  > > > 2. octave-3.7.2+ VS*****************************
> >>  > > > >> fid = 
> > fopen("TestData.dat","r");
> >>  > > > >> s = fscanf(fid,'%c');
> >>  > > > warning: range error for conversion to character value
> >>  > > > >> fclose(fid);
> >>  > > > >> s = regexprep(s,'0','2');
> >>  > > > >> clear all
> >>  > > > >> fid = 
> > fopen("TestData.dat","r");
> >>  > > > >> s = fscanf(fid,'%c');
> >>  > > > error: out of memory or dimension too large for Octave's 
> > index type
> >>  > > > 
> > ********************************************************************
> >>  > > > 
> >>  > > > 3.Octave-3.6.2 
> > Cygwin**********************************************
> >>  > > > >> fid = 
> > fopen("TestData.dat","r");
> >>  > > > >> s = fscanf(fid,'%c');
> >>  > > > >> fclose(fid);
> >>  > > > >> s = regexprep(s,'0','2');
> >>  > > > >> clear all
> >>  > > > >> fid = 
> > fopen("TestData.dat","r");
> >>  > > > >> s = fscanf(fid,'%c');
> >>  > > > >> fclose(fid);
> >>  > > > >> s = regexprep(s,'0','2');
> >>  > > > 
> > ********************************************************************
> >>  > > > 
> >>  > > > For me, both VS and MinGW version caused problem. Cygwin 
> > version seemed to work correctly.
> >>  > > > 
> >>  > > > Regards
> >>  > > > 
> >>  > > > Tatsuro
> >>  > > 
> >>  > > 
> >>  > > I have also tested on 
> >>  > > octave-3.2.4 and 3.4.3 MinGW version.
> >>  > > 
> >>  > > 4. octave-3.2.4 and 3.4.3 MinGW *****************************
> >>  > > >> fid = fopen("TestData.dat","r");
> >>  > > >> s = fscanf(fid,'%c');
> >>  > > warning: range error for conversion to character value
> >>  > > >> fclose(fid);
> >>  > > >> s = regexprep(s,'0','2');
> >>  > > >> clear all
> >>  > > >> fid = fopen("TestData.dat","r");
> >>  > > >> s = fscanf(fid,'%c');
> >>  > > warning: range error for conversion to character value
> >>  > > >> fclose(fid);
> >>  > > >> s = regexprep(s,'0','2');
> >>  > > **************************************************
> >>  > > 
> >>  > > The problem seemed to be happened on octave-3.6 or later on 
> > native windows.
> >>  > > 
> >>  > > Regards
> >>  > > 
> >>  > > Tatsuro 
> >>  > 
> >>  > Sorry for successive post.
> >>  > 
> >>  > I have tested the following  :
> >>  > 
> >>  > %*********************************
> >>  > s="0123456789";
> >>  > for i=1:21, s=cstrcat(s,s);,endfor
> >>  > save TestData.dat s
> >>  > load TestData.dat
> >>  > s = regexprep(s,'0','2');
> >>  > clear all
> >>  > load TestData.dat
> >>  > s = regexprep(s,'0','2');
> >>  > %*********************************
> >>  > 
> >>  > The above worked correctly on octave-3.2.4 (MinGW), 3.4.3 (MinGW), 
> > 3.6.2 (MinGW,VS, Cygwin) and 3.7.2+ (VS).
> >>  > 
> >>  > I think that the problem does not rely on regexprep but on fscanf for 
> > large data.
> >>  > 
> >>  > Regards
> >>  > 
> >>  > Tatsuro
> >>  > 
> >>  Thank you very much for the fast confirmation of the same problem. 
> >> Actually 
> > it must be a strange combination of both I tested:
> >> 
> >>  octave-3.7.2+ VS*****************************
> >>  fid = fopen("TestData.dat","r");
> >>  s = fscanf(fid,'%c');
> >>  warning: range error for conversion to character value
> >>  fclose(fid);
> >>  clear all
> >>  fid = fopen("TestData.dat","r");
> >>  s = fscanf(fid,'%c');
> >>  warning: range error for conversion to character value
> >>  fclose(fid);
> >>  clear all
> >>  fid = fopen("TestData.dat","r");
> >>  s = fscanf(fid,'%c');
> >>  warning: range error for conversion to character value
> >>  fclose(fid);
> >>  clear all
> >>  ********************************************************************
> >> 
> >>  So it can't be fscanf only. On the other hand I tested as well (because 
> > your last example the second regexprep does nothing, as all 0 were already 
> > exchanged)
> >> 
> >>  octave-3.7.2+ VS*****************************
> >>  fid = fopen("TestData.dat","r");
> >>  s = fscanf(fid,'%c');
> >>  warning: range error for conversion to character value
> >>  fclose(fid);
> >>  s = regexprep(s,'0','2');
> >>  s = regexprep(s,'1','2');
> >>  s = regexprep(s,'2','3');
> >>  error: out of memory or dimension too large for Octave's index type
> >>  ********************************************************************
> >> 
> >>  So it clearly also depends on how many replacements are performed within 
> > the regexprep. (same fails as well in 3.6.0_gcc4.6.2 and 3.2.4_gcc4.4.0)
> >>  As something like this always works:
> >> 
> >>  octave-3.7.2+ VS*****************************
> >>  fid = fopen("TestData.dat","r");
> >>  s = fscanf(fid,'%c');
> >>  warning: range error for conversion to character value
> >>  fclose(fid);
> >>  s = ctsrcat(s,"aaaaaaaaa", s);
> >>  s = regexprep(s,'a','2');
> >>  clear all
> >>  fid = fopen("TestData.dat","r");
> >>  s = fscanf(fid,'%c');
> >>  warning: range error for conversion to character value
> >>  fclose(fid);
> >>  clear all
> >>  ********************************************************************
> >> 
> >>  Kind regards
> >>  Christoph
> > 
> > You are right.
> > #**************************************
> > s="0123456789";
> > for i=1:21, s=cstrcat(s,s);,endfor
> > 
> > s = regexprep(s,'0','2'); 
> > s = regexprep(s,'1','2'); 
> > s = regexprep(s,'2','3'); 
> > #**************************************
> > 
> > The above caused 'out of memory ...' or 'memory exhausted ...' 
> > on octave 3.7.2+(VS),
> > 3.6.2(VS, MinGW), 3.4.3 (MinGW) and 3.2.4 (MinGW).
> > 
> > But for cygwin version octave(3.6.2), the above test did not cause memory 
> > error.
> > 
> > I think that it is better that you resist this strange behavior of 
> > regexprep as 
> > a bug to the bug tracker.
> > 
> > Regards
> > 
> > Tatsuro
> > _______________________________________________
> 
> 
> I've tried the above code on self-built octave-3.6.4 under Linux.
> 
> Memory consumption constantly grows, I killed the session when memory 
> consumption reached 2.5G.
> 
> I think it's a bug, and since I can reproduce the failure under Linux, the 
> bug is not OS-specific.
> 
> Regards,
>   Sergei.
> 
>