help-octave
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: 0.3*3 == 0.9?


From: Marc Weber
Subject: Re: 0.3*3 == 0.9?
Date: Wed, 25 Aug 2010 16:21:50 +0200
User-agent: Sup/git

Hi Michael,

take *any* book and learn about internal number representation.
Numbers are represented in dual format rather than decimal.
This means that 0.9 (which looks nice in decimal) can be something like
.111111 (repeat infinitely) in dual representation (something like 1/3
in decimal which is 0.33333... )

Thus 0.9 != 0.9.

Conclusion: Never ever use strict equality for comparison. Always use
abs(number1 - number2) < 0.000001 or something like that.

Probably octave has functions for that - don't know it well enough.
Digg around in existing code - maybe you find how its done there.

I'm not an Octave expert.

Marc Weber


reply via email to

[Prev in Thread] Current Thread [Next in Thread]