[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: Is this a numerical problem?
From: |
Lukas Reichlin |
Subject: |
Re: Is this a numerical problem? |
Date: |
Thu, 19 Aug 2010 14:32:49 +0200 |
On 19.08.2010, at 13:16, Jaroslav Hajek wrote:
> On Thu, Aug 19, 2010 at 12:52 PM, Lukas Reichlin
> <address@hidden> wrote:
>> Hi Jaroslav
>>
>> I realized that (in most cases, see below) floats are rounded, not truncated
>> such that -4.00004 and -3.99995 are both displayed as -4.0000. Therefore you
>> can't tell whether -4.0000 is less or greater than -4.
>>
>> octave:15> 3.9999
>> ans = 3.9999
>> octave:16> 3.99995
>> ans = 4.0000
>> octave:17> 0.99995
>> ans = 0.99995
>> octave:18> 0.999995
>> ans = 0.99999 # I'm expecting 1.00000 here
>> octave:19> 0.9999995
>> ans = 1.00000
>> octave:20> 0.999996
>> ans = 1.00000 # compare this to command 18
>>
>> Regards,
>> Lukas
>>
>
> Exactly. Which brings us back to your observation:
>
> octave:1> format long
> octave:2> a = (10 * rem (-12/5, 1))
> a = -4.00000000000000
> octave:3> a - -4.00000000000000
> ans = 8.88178419700125e-16
> octave:4> -3.99999999999999 - a
> ans = 9.32587340685131e-15
>
> which is an explanation.
>
> --
> RNDr. Jaroslav Hajek, PhD
> computing expert & GNU Octave developer
> Aeronautical Research and Test Institute (VZLU)
> Prague, Czech Republic
> url: www.highegg.matfyz.cz
Thanks for your patience. I consider the display of result 54 as an
inconsistency or a (cosmetic) bug because round (0.5) is 1. Rounding up starts
at 0.50, not at 0.51.
(Only the first digit after truncation is significant for rounding)
octave:57> 0.999996
ans = 1.00000
octave:53> 0.999995
ans = 0.99999 # this should be 1.00000 like the previous result
octave:54> 0.9999951
ans = 1.00000
octave:55> 0.9999950
ans = 0.99999 # this should be rounded up too
octave:56> 0.9999949
ans = 0.99999
Results from the "dark side":
>> 0.99995
ans =
1.0000 % this is correct
>> 0.999949
ans =
0.9999
>> 0.999950
ans =
1.0000
>> 0.999951
ans =
1.0000
>> 0.99996
ans =
1.0000
>>
Cheers,
Lukas
- Is this a numerical problem?, Lukas Reichlin, 2010/08/19
- Re: Is this a numerical problem?, Jaroslav Hajek, 2010/08/19
- Re: Is this a numerical problem?, Lukas Reichlin, 2010/08/19
- Message not available
- Re: Is this a numerical problem?, Jaroslav Hajek, 2010/08/19
- Re: Is this a numerical problem?, Lukas Reichlin, 2010/08/19
- Re: Is this a numerical problem?, Sergei Steshenko, 2010/08/19
- Re: Is this a numerical problem?, Jaroslav Hajek, 2010/08/19
- Re: Is this a numerical problem?, Miguel Bazdresch, 2010/08/19
- Re: Is this a numerical problem?,
Lukas Reichlin <=
- Re: Is this a numerical problem?, Jaroslav Hajek, 2010/08/19
- Re: Is this a numerical problem?, Lukas Reichlin, 2010/08/19
- Re: Is this a numerical problem?, George Barrick, 2010/08/19
- Re: Is this a numerical problem?, Jordi Gutiérrez Hermoso, 2010/08/19
- Re: Is this a numerical problem?, Ozzy Lash, 2010/08/19
- Re: Is this a numerical problem?, Jordi Gutiérrez Hermoso, 2010/08/19
- Re: Is this a numerical problem?, Judd Storrs, 2010/08/19
- Re: Is this a numerical problem?, Jordi Gutiérrez Hermoso, 2010/08/19
- Re: Is this a numerical problem?, Lukas Reichlin, 2010/08/19
- Re: Is this a numerical problem?, Judd Storrs, 2010/08/19