[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: Why does this happen
|
From: |
A. Scottedward Hodel |
|
Subject: |
Re: Why does this happen |
|
Date: |
Tue, 13 Oct 1998 10:29:15 -0500 |
> From address@hidden Tue Oct 13 10:24:37 1998
>
>
> B =
>
> 7973.60000
> 0.96500
> 2.71400
>
> octave:5> B
> B =
>
> 7973.600000000000364
> 0.965000000000000
> 2.714000000000000
>
>
> Why is B(1) now suddenly 7973.600000000000364 and not
> 7973.600000000000000 ?
Because double precision arithmetic has approx 16 digits of precision,
and because numbers are stored using (usually) base 2, not base 10.
Finite-length base 10 fractions are often repeating numbers in binary.
This is inherent in any floating-point computation.