Re: Catastrophic Cancellation

[Rob Mahurin]

On Jul 3, 2008, at 8:50 AM, Sergei Steshenko wrote:
> octave:1> (0.3 - 0.2 - 0.1)/1e-16
> ans = -0.27756
>
> - I do not understand what you wanted to say by your reply.
>
> I.e. I do not understand what/where the problem is.


There have been a few questions on this list in the past couple weeks  
phrased as "how can I get more precision from Octave," but actually  
problems of floating-point truncation in numerical analysis.  One was  
a question about a p-value like 1 - (1e-30).  One was perhaps an  
arbitrary precision question?  This seemed to prompt AK to post his  
example of the difference between the three expressions

        (1-cos(x)) ./ x.^2;
        1./ x.^2 - cos(x) ./ x.^2; and
        (.5 - x.^2 / 24)

as computed by octave (or anything else) on the interval between 0  
and 1e-7.  It really is a pretty example of the sorts of things you  
see when you hit a truncation error problem.  But it's only Octave- 
specific in the sense that Octave is the first introduction to  
numerical analysis for many people.

Cheers,
Rob

--
Rob Mahurin
Dept. of Physics & Astronomy
University of Tennessee         phone: 865 207 2594
Knoxville, TN 37996             email: address@hidden