Re: floating point arithmetic problems

[Ozzy Lash]

On Tue, Oct 24, 2017 at 3:27 PM, AG <address@hidden> wrote:

I thought I understood floating point problems but I guess I don't and now I
have a lot of code that is acting strangely given my understanding.
...
Is there a simple and consistent way to avoid problems like these? I would
have though that with only 2 decimal places floating point problems(which is
what I assume this is) wouldn't be a problem. Thanks for the help.

It really comes down to how numbers are represented in floating point.  Effectively they are stored as an exponent of 2 and a mantissa, so that the number being represented is mantissa*2^exponent. Some numbers look simple in base 10, but will give a lot of digits in the mantissa, so when you do addition and subtractions you get a lot of binary digits.

Anyway, the  way I have usually seen used to compare for floating point "equality" is

instead of a==b use abs(a-b)<tol

unfortunately that means you need to come up with a tolerance value.  I have seen people use a tolerance of "eps" which is the machine epsilon, which is the amount that the number 1 and the next representable number are away from each other.  This should work ok if the numbers are near 1, but would fail if the numbers are very large or very small.  A simple modification would be to use eps(a), or maybe eps(max(a,b)).  I think this should work in most cases.  There are probably other possibilities, but I think that for most cases

(abs(a-b)<eps(a))

would be a good general replacement.

Bill