Re: Problem with the floor function

[Michael Hanke]

Hej,

You ran in the interesting world of rounding errors. All computations
in octave (and matlab) are done in double precision IEEE numbers.
They use the base 2 while we use conveniently decimal numbers for
input/output representation. You are computing something like
15/0.01. But 0.01 does not have a finite representation in the binary
system. So you will not obtain 1500 but a rounded result. Your octave
session indicates that it is slightly lower than 1500.

This problem is very common in defining a grid in floating point
numbers:

a=...;
b=...;
h=(b-a)/m;
x=a:h:b;

gives often not the desired result. I am using instead

x=a:h:b+h/2;

in order to avoid a surprise when considering the length of x.

Hope this helps.

Michael




On Tue, 19 Jan 1999, Daniel Tourde wrote:

> Hello,
> 
> I have discovered something strange with the floor function. I suspect a
> bug but where ? John W. Eaton would bet that floor is correct, but that
> the display routines in the C/C++ I/O library are doing
> something slightly different when they display values that are very
> near but just less than a representable integer.
> 
> octave:1> floor(46.000)
> ans = 46
> octave:2> k = 0.03
> k = 0.030000
> octave:3> kmin=0
> kmin = 0
> octave:4> kstep = 0.01
> kstep = 0.010000
> octave:5> modes=15
> modes = 15
> octave:6> (1+(k-kmin)*modes/kstep)
> ans = 46.000
> octave:7> floor(1+(k-kmin)*modes/kstep)
> ans = 45                                     <-  It should be 46, no ?
> octave:8> (k-kmin)*modes
> ans = 0.45000
> octave:9> (k-kmin)*modes/kstep
> ans = 45.000
> octave:10> floor((k-kmin)*modes/kstep)
> ans = 44                                     <-  It should be 45, no ?
> 
> Strange isn't it ?
> Does anyone has an idea of where it comes from and how it could be
> solved ?
> 
> Thanks in advance
> 
>                                 Daniel
> -- 
> ***********************************************************************
> Daniel TOURDE                                   E-mail : address@hidden
> The Aeronautical Research Institute of Sweden   Tel : +46 8 55 54 93 44
> P.O. Box 11021 S-161 11 BROMMA, Sweden          Fax : +46 8    25 34 81
> ***********************************************************************
> 
> 

+---------------------------------------------------------------+
|  Michael Hanke                Royal Institute of Technology   |
|                               NADA                            |
|                               S-10044 Stockholm               |
|                               Sweden                          |
+---------------------------------------------------------------+
|  Visiting address:            Lindstedtsvaegen 3              |
|  Phone:                       + (46) (8) 790 6278             |
|  Fax:                         + (46) (8) 790 0930             |
|  Email:                       address@hidden               |
|                               address@hidden       |
+---------------------------------------------------------------+