determinant of integer values

[John W. Eaton]

On  1-Mar-2006, Peter Lempel Søndergaard wrote:

| the following code
| 
| A=[1 0; 1 12];
| det(A)
| 
| produces:
| 
| ans = 12.000
| 
| instead of the better result ans = 12 .
| 
| I am using Fedora 4 with the lastest rpms installed, Octave 2.1.72 and
| Octave-forge.
| 
| This is slightly annoying, because gcd will not work with 12.000, so I
| have to round the number. Matlab gets this one correctly.

Bug reports belong on the address@hidden list so I'm copying this
message there.  If you reply, please remove the help list from the
list of recipients.

The problem is that Octave stores the value of the determinant
internally as det = a*10^b so it does not overflow or underflow
unnecessarily.  For the scripting language, we always convert to a
double value, so DET can produce Inf, but in C++ code you can look at
the values of A and B to find a better estimate of the value.

Perhaps the best fix is to change the internal representation of the
determinant to c*2^d instead.  I think that should work for binary
machines (and who has a ternary or decimal computer these days
anyway?).

It would be nice to preserve the base-10 meaning of the current
coefficient and exponent methods in the determinant classes.  Does
anyone have a good way to efficiently and accurately convert from
c*2^d to a*10^b?

Thanks,

jwe



-------------------------------------------------------------
Octave is freely available under the terms of the GNU GPL.

Octave's home on the web:  http://www.octave.org
How to fund new projects:  http://www.octave.org/funding.html
Subscription information:  http://www.octave.org/archive.html
-------------------------------------------------------------