help-octave
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Problem with the floor function


From: Daniel Tourde
Subject: Problem with the floor function
Date: Tue, 19 Jan 1999 08:17:44 +0100

Hello,

I have discovered something strange with the floor function. I suspect a
bug but where ? John W. Eaton would bet that floor is correct, but that
the display routines in the C/C++ I/O library are doing
something slightly different when they display values that are very
near but just less than a representable integer.

octave:1> floor(46.000)
ans = 46
octave:2> k = 0.03
k = 0.030000
octave:3> kmin=0
kmin = 0
octave:4> kstep = 0.01
kstep = 0.010000
octave:5> modes=15
modes = 15
octave:6> (1+(k-kmin)*modes/kstep)
ans = 46.000
octave:7> floor(1+(k-kmin)*modes/kstep)
ans = 45                                     <-  It should be 46, no ?
octave:8> (k-kmin)*modes
ans = 0.45000
octave:9> (k-kmin)*modes/kstep
ans = 45.000
octave:10> floor((k-kmin)*modes/kstep)
ans = 44                                     <-  It should be 45, no ?

Strange isn't it ?
Does anyone has an idea of where it comes from and how it could be
solved ?

Thanks in advance

                                Daniel
-- 
***********************************************************************
Daniel TOURDE                                   E-mail : address@hidden
The Aeronautical Research Institute of Sweden   Tel : +46 8 55 54 93 44
P.O. Box 11021 S-161 11 BROMMA, Sweden          Fax : +46 8    25 34 81
***********************************************************************



reply via email to

[Prev in Thread] Current Thread [Next in Thread]