[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
## floating point arithmetic problems

**From**: |
AG |

**Subject**: |
floating point arithmetic problems |

**Date**: |
Tue, 24 Oct 2017 13:27:17 -0700 (MST) |

I thought I understood floating point problems but I guess I don't and now I
have a lot of code that is acting strangely given my understanding.
EG.
0.2-0.1==0.1
ans = 1 #GREAT!
1.2-1.1==0.1
ans = 0 #HUH?
Here's one concrete example from some code I was playing with. all this code
does is rebuilding the decimal column using start:increment:end syntax and
compare the old column to the new.
test = [
96.02,3827;
96.03,5341;
96.04,6107;
96.05,7134;
96.06,8706;
96.07,12367;
96.08,13971;
96.09,16900;
96.10,15700;
96.11,18791;
96.12,22168;
96.13,21973;
96.14,21800;
96.15,19626;
96.16,17601;
96.17,16496;
96.18,14921;
96.19,10819;
96.20,9087;
96.21,6056;
96.22,5071;
96.23,4311];
newfirstcol = (min(x):0.01:max(x))'; #should have same values as tests 1st
column..
sum(ismember(newfirstcol,test(:,1))) #matches only 16 times....
Is there a simple and consistent way to avoid problems like these? I would
have though that with only 2 decimal places floating point problems(which is
what I assume this is) wouldn't be a problem. Thanks for the help.
--
Sent from: http://octave.1599824.n4.nabble.com/Octave-General-f1599825.html

**floating point arithmetic problems**,
*AG* **<=**