[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: floating point arithmetic problems
From: |
AG |
Subject: |
Re: floating point arithmetic problems |
Date: |
Wed, 25 Oct 2017 09:49:33 -0700 (MST) |
Ozzy,Gordon & Allen thanks a bunch for taking the time to explain to me. It
was a HUUUGE help.
Given your inputs I ended up creating a 'rounding' function that used the
idea of tolerance and cleaving decimal places.
As an example:
For rounding to two decimals myRound(num) does something like:
num = num + .001;
roundedNum = floor( NUM*100 ) / 100;
Everything seems to be working perfectly now. Thanks again for your help
and time.
--
Sent from: http://octave.1599824.n4.nabble.com/Octave-General-f1599825.html