[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: floating point arithmetic problems

From: AG
Subject: Re: floating point arithmetic problems
Date: Wed, 25 Oct 2017 09:49:33 -0700 (MST)

Ozzy,Gordon & Allen thanks a bunch for taking the time to explain to me.   It
was a HUUUGE help.  

Given your inputs I ended up creating a 'rounding' function that used the
idea of tolerance and cleaving decimal places.
As an example:
For rounding to two decimals myRound(num) does something like:
num = num + .001;
roundedNum = floor( NUM*100 ) / 100;

Everything seems to be working perfectly now.  Thanks again for your help
and time.

Sent from:

reply via email to

[Prev in Thread] Current Thread [Next in Thread]