[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
## RE: floating point arithmetic problems

**From**: |
Windhorn, Allen E [INDAUTO/LS/MKT] |

**Subject**: |
RE: floating point arithmetic problems |

**Date**: |
Tue, 24 Oct 2017 20:58:09 +0000 |

AG,
>* -----Original Message-----*
>* From: Help-octave [mailto:help-octave-*
>* *
>* I thought I understood floating point problems but I guess I don't and now I*
>* have a lot of code that is acting strangely given my understanding.*
>* EG.*
>* 0.2-0.1==0.1*
>* ans = 1 #GREAT!*
>* *
>* 1.2-1.1==0.1*
>* ans = 0 #HUH?*
>* *
>* Here's one concrete example from some code I was playing with. all this code*
>* does is rebuilding the decimal column using start:increment:end syntax and*
>* compare the old column to the new.*
>* *
>* test = [*
>* 96.02,3827;*
>* ...*
>* 96.23,4311];*
>* newfirstcol = (min(x):0.01:max(x))'; #should have same values as tests 1st*
>* column..*
>* sum(ismember(newfirstcol,test(:,1))) #matches only 16 times....*
>* *
>* Is there a simple and consistent way to avoid problems like these? I would*
>* have though that with only 2 decimal places floating point problems(which is*
>* what I assume this is) wouldn't be a problem. Thanks for the help.*
As I have been taught, you should NEVER use an equality comparison
on floating-point numbers. At least, compare with +/- eps of one of
the values.
The 2 decimal places you show become an infinite repeating binary
fraction when converted. I don't know why some work and others
don't, but if you did the conversion by hand you would probably be
enlightened.
0.1 decimal is 0.000110011001100110011.... binary for instance.
Regards,
Allen