RE: floating point arithmetic problems

[Windhorn, Allen E [INDAUTO/LS/MKT]]

AG,

> -----Original Message-----
> From: Help-octave [mailto:help-octave-
> 
> I thought I understood floating point problems but I guess I don't and now I
> have a lot of code that is acting strangely given my understanding.
> EG.
> 0.2-0.1==0.1
> ans = 1    #GREAT!
> 
> 1.2-1.1==0.1
> ans = 0    #HUH?
> 
> Here's one concrete example from some code I was playing with. all this code
> does is rebuilding the decimal column using start:increment:end syntax and
> compare the old column to the new.
> 
> test = [
> 96.02,3827;
> ...
> 96.23,4311];
> newfirstcol = (min(x):0.01:max(x))'; #should have same values as tests 1st
> column..
> sum(ismember(newfirstcol,test(:,1))) #matches only 16 times....
> 
> Is there a simple and consistent way to avoid problems like these? I would
> have though that with only 2 decimal places floating point problems(which is
> what I assume this is) wouldn't be a problem. Thanks for the help.

As I have been taught, you should NEVER use an equality comparison
on floating-point numbers.  At least, compare with +/- eps of one of
the values.

The 2 decimal places you show become an infinite repeating binary
fraction when converted.  I don't know why some work and others
don't, but if you did the conversion by hand you would probably be
enlightened.

0.1 decimal is 0.000110011001100110011.... binary for instance.

Regards,
Allen