help-octave
[Top][All Lists]

## Octave "precision"/sig. digits ?

 From: Daniel Friedman Subject: Octave "precision"/sig. digits ? Date: Sun, 13 Jul 1997 01:53:01 -0400 (EDT)

```Summary:      Is there a way to get Octave to regard x, where 0<x<eps,
as > 0 (where eps is the Octave built-in variable)?

WARNING:      The rest of this mail is long.
APPECIATION:  Thanks for taking the time to read it.

Details:
----------
I am trying to compute something which, by its nature, it seems
Octave has a litle trouble with.  In particular, I wish to compute

M
---
\
f(M,p) =  /   bincoeff(M,i) * ((-1)^(i+1)) / (1-p^i)
---
i=1

where M is a natural number and 0<p<0.25 .

For example, Octave tells me f(25, 0.1)  = 2.177866.  Good.
But, Octave gives me this:

f(47, 0.1) = 2.419922
f(48, 0.1) = 1.427734
f(49, 0.1) = 4.437500
f(50, 0.1) = -2.554688

Now from the situation of interest I know that 0 < f(M, p) < f(M+1,p),
so f(49, 0.1) and f(50, 0.1) are both incorrect.

Now p^i is particularly small as i grows, so I suspect[ed] that Octave
was just calling it 0 (zero) once i became sufficiently large.

[ See:
octave:33> format long e
octave:34> 1-(0.1 ^ 14)
ans =  9.99999999999990e-01
octave:35> 1-(0.1 ^ 15)
ans =  9.99999999999999e-01
octave:36> 1-(0.1 ^ 16)
ans =  1.00000000000000e-00
octave:37> 1-(0.1 ^ 17)
ans =  1.00000000000000e+00
octave:38> eps
eps =  2.22044604925031e-16

Now tell me why the answers for 1-(0.1 ^ 16) and 1-(0.1 ^ 17) are
slightly different? ]

Hence I performed the same calculation  using the "arbitrary precision
calculator", 'bc' (version 1.03 (Nov 2, 1994)) . I set "scale" [ quoting
from the man page for bc, "scale is the the total number of decimal digits
after the decimal point" and, if you set it, "scale defines how some
operations use digits  after the decimal point."] to 30, and found, more
plausibly,

f(47, 0.1) = 2.420551
f(48, 0.1) = 2.428558
f(49, 0.1) = 2.436431
f(50, 0.1) = 2.444177

I'm not so concerned that the Octave results, when correct, differ
slightly from the bc results.  I *am* concerned that, for some inputs,
Octave doesn't give the correct answer.

---> What am I doing wrong/what should I do?

---------------

Many thanks for helping me enjoy Octave,
--dan

---------------------------------------------------------------------------