[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
precision of calculations
From: |
Robert Butora |
Subject: |
precision of calculations |
Date: |
Fri, 23 Jul 1999 15:27:44 +0300 |
Does anybody know how the precision of calculation
(not output_precision !) is controlled in Octave ?
I've calculated some matrix where I expected results
to be 1 or 0. Instead of 0 I've got values in order
of 1e-16 ... 1e-18.
Should I chase an error in my calculation or does it
have something to do with precision of calc. ?
Could anybody give some general thoughts on precision in Octave ?
Thanks
Robert
---------------------------------------------------------------------
Octave is freely available under the terms of the GNU GPL. To ensure
that development continues, see www.che.wisc.edu/octave/giftform.html
Instructions for unsubscribing: www.che.wisc.edu/octave/archive.html
---------------------------------------------------------------------
- precision of calculations,
Robert Butora <=