[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
## Re: trying to understand why I get 2.2204e-16 rather than 0

**From**: |
James Sherman Jr. |

**Subject**: |
Re: trying to understand why I get 2.2204e-16 rather than 0 |

**Date**: |
Wed, 3 Aug 2016 21:46:21 -0400 |

On Wed, Aug 3, 2016 at 8:41 PM, naryana.shankara
<address@hidden> wrote:
>* If I enter cot (pi/4)-1 I get 2.2204e-16*
>* From my trig days I remember cot (pi/4) is 1 and 1-1 is 0*
>* Octave also confirms cot (pi/4) is 1 and 1-1 is 0*
>
>* so why does Octave spit out 2.2204e-16 when cot(pi/4)-1 is entered?*
>
>
>
>* --*
>* View this message in context: *
>* http://octave.1599824.n4.nabble.com/trying-to-understand-why-I-get-2-2204e-16-rather-than-0-tp4678993.html*
>* Sent from the Octave - General mailing list archive at Nabble.com.*
>
>* _______________________________________________*
>* Help-octave mailing list*
>* address@hidden*
>* https://lists.gnu.org/mailman/listinfo/help-octave*
Some background reading is probably the best:
https://en.wikipedia.org/wiki/Floating_point#Representable_numbers.2C_conversion_and_rounding
Note the reason you're getting isn't Octave so much as a property of
floating point numbers.
Really the whole article is pretty good, but it boils down to
essentially that representing most numbers is an approximation because
of finite memory with decimal representation, let alone irrational
ones like pi. This isn't even getting into things like how trig
functions are calculated. To show that it is even more fundamental
than that, try
>*> (1-1/3)-2/3*
and see the result you get.
James Sherman