help-octave
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Weird precision in matrix


From: Jordi Gutiérrez Hermoso
Subject: Re: Weird precision in matrix
Date: Sat, 19 Nov 2011 11:10:42 -0500

On 19 November 2011 06:02, studentpol <address@hidden> wrote:

> When im running it i recieve:
> A =
>
>  Columns 1 through 3:
>
>     0.000000000000000   583.333459999999945     0.000000000000000
>
>  Columns 4 and 5:
>
>     0.000000000000000     0.000000000000000
>
> Why it looks like this? I wanted just precisly 583.33346. How to do it?

You cannot represent 583.33346 exactly in binary. What you are seeing
is the closest binary approximation (IEEE 754 floating point) in
decimal. Either you learn to live with the fact that 0.3 - 0.2 - 0.1
isn't zero and you write programs that carefully account for rounding
errors, or you use much slower arbitrary-precision arithmetic as
provided by the vpa function in the symbolic package. Hardware floats
like Octave's native type are much faster than a software
implementation of arbitrary precision.

It is the way of things...

This is a very long and detailed discussion of this subject:

    http://download.oracle.com/docs/cd/E19957-01/806-3568/ncg_goldberg.html

If you want a briefer introduction, the Wikipedia article furnishes it:

    http://en.wikipedia.org/wiki/Floating_point_arithmetic

HTH,
- Jordi G. H.


reply via email to

[Prev in Thread] Current Thread [Next in Thread]