help-octave
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Is this a numerical problem?


From: Judd Storrs
Subject: Re: Is this a numerical problem?
Date: Thu, 19 Aug 2010 11:51:39 -0400

2010/8/19 Jordi Gutiérrez Hermoso <address@hidden>
I believe the inconsistency is how 0.999995 is displayed in each with
"format short". Octave displays 0.99999 and Matlab displays 1.0000,
from what I can read in the bug report.

I don't understand this comparison. Matlab is displaying four digits after the decimal whereas octave is displaying five. If octave displayed four digits it would also display 1.0000. 

Why is Matlab more correct? The inaccuracy of the Matlab representation is +0.000005 and the inaccuracy of the Octave representation is -0.000005. Why is one "correct" and the other wrong?

Last time I checked GCC has very excellent binary to decimal conversion routines.


--judd

reply via email to

[Prev in Thread] Current Thread [Next in Thread]