help-octave
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Question about Range class


From: Robert A. Macy
Subject: Re: Question about Range class
Date: Fri, 25 Nov 2005 12:59:07 -0800

Interesting.  Didn't know that.  

       - Robert -

On Fri, 25 Nov 2005 15:29:51 -0500 (EST)
 Przemek Klosowski <address@hidden> wrote:
> 
>    As an accuracy test I tried    sqrt(2)*sqrt(2)-2
> 
>    My TI calculator came back with...      1e-11
>     ...error
>    octave came back with...             4.44e-16
>      ...interesting.  
> 
> You do know why this is, right? the error in such
> calculations is on the
> order of a Least Significant Digit in whatever numbering
> system your machine
> uses in the calculation. Octave uses double precision
> IEEE floating point,
> which has 52 mantissa binary digits, so the error is on
> the order of 1/2^52
> (octave variable 'eps', equal to ~ 2.220446e-16). TI
> calculators use a different
> floating point format, BCD-based, if I remember
> correctly--and I'd guess it
> uses 11 significant decimal digits.
> 



-------------------------------------------------------------
Octave is freely available under the terms of the GNU GPL.

Octave's home on the web:  http://www.octave.org
How to fund new projects:  http://www.octave.org/funding.html
Subscription information:  http://www.octave.org/archive.html
-------------------------------------------------------------



reply via email to

[Prev in Thread] Current Thread [Next in Thread]