help-octave
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Octave "precision"/sig. digits ?


From: Daniel Friedman
Subject: Octave "precision"/sig. digits ?
Date: Sun, 13 Jul 1997 01:53:01 -0400 (EDT)

Summary:      Is there a way to get Octave to regard x, where 0<x<eps, 
                as > 0 (where eps is the Octave built-in variable)?

WARNING:      The rest of this mail is long.
APPECIATION:  Thanks for taking the time to read it.

Details:
----------
I am trying to compute something which, by its nature, it seems
Octave has a litle trouble with.  In particular, I wish to compute

            M
           ---
           \   
 f(M,p) =  /   bincoeff(M,i) * ((-1)^(i+1)) / (1-p^i)
           ---
           i=1

where M is a natural number and 0<p<0.25 .

For example, Octave tells me f(25, 0.1)  = 2.177866.  Good.
But, Octave gives me this:

f(47, 0.1) = 2.419922
f(48, 0.1) = 1.427734
f(49, 0.1) = 4.437500
f(50, 0.1) = -2.554688


Now from the situation of interest I know that 0 < f(M, p) < f(M+1,p),
so f(49, 0.1) and f(50, 0.1) are both incorrect.

Now p^i is particularly small as i grows, so I suspect[ed] that Octave
was just calling it 0 (zero) once i became sufficiently large.

[ See:
octave:33> format long e 
octave:34> 1-(0.1 ^ 14)
ans =  9.99999999999990e-01
octave:35> 1-(0.1 ^ 15)
ans =  9.99999999999999e-01
octave:36> 1-(0.1 ^ 16)
ans =  1.00000000000000e-00
octave:37> 1-(0.1 ^ 17)
ans =  1.00000000000000e+00
octave:38> eps
eps =  2.22044604925031e-16

Now tell me why the answers for 1-(0.1 ^ 16) and 1-(0.1 ^ 17) are
slightly different? ]


Hence I performed the same calculation  using the "arbitrary precision
calculator", 'bc' (version 1.03 (Nov 2, 1994)) . I set "scale" [ quoting
from the man page for bc, "scale is the the total number of decimal digits
after the decimal point" and, if you set it, "scale defines how some
operations use digits  after the decimal point."] to 30, and found, more
plausibly,

f(47, 0.1) = 2.420551
f(48, 0.1) = 2.428558
f(49, 0.1) = 2.436431
f(50, 0.1) = 2.444177

I'm not so concerned that the Octave results, when correct, differ
slightly from the bc results.  I *am* concerned that, for some inputs,
Octave doesn't give the correct answer.

---> What am I doing wrong/what should I do?

---------------

Many thanks for helping me enjoy Octave,
--dan

---------------------------------------------------------------------------
 Daniel Friedman    address@hidden    http://www.isr.umd.edu/~danielf

 Center for Satellite and Hybrid Communication Networks
 Institute for Systems Research
 University of Maryland
 College Park, MD 20742



reply via email to

[Prev in Thread] Current Thread [Next in Thread]