[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: trying to understand why I get 2.2204e-16 rather than 0

From: James Sherman Jr.
Subject: Re: trying to understand why I get 2.2204e-16 rather than 0
Date: Wed, 3 Aug 2016 21:46:21 -0400

On Wed, Aug 3, 2016 at 8:41 PM, naryana.shankara
<address@hidden> wrote:
> If I enter cot (pi/4)-1 I get 2.2204e-16
> From my trig days I remember cot (pi/4) is 1 and 1-1 is 0
> Octave also confirms cot (pi/4) is 1 and 1-1 is 0
> so why does Octave spit out 2.2204e-16 when cot(pi/4)-1 is entered?
> --
> View this message in context: 
> Sent from the Octave - General mailing list archive at
> _______________________________________________
> Help-octave mailing list
> address@hidden

Some background reading is probably the best:
Note the reason you're getting isn't Octave so much as a property of
floating point numbers.

Really the whole article is pretty good, but it boils down to
essentially that representing most numbers is an approximation because
of finite memory with decimal representation, let alone irrational
ones like pi.  This isn't even getting into things like how trig
functions are calculated.  To show that it is even more fundamental
than that, try

>> (1-1/3)-2/3

and see the result you get.

James Sherman

reply via email to

[Prev in Thread] Current Thread [Next in Thread]