Re: uint64 typecast can't be initialised with more than 15 decimal positions

[Yury T.]

Thank you very much, indeed.

However, I must comment on the document quoted (and yes, I can and will edit
it, just as 'anybody' can -- later).

First thing, such limitations aren't meant to be found in FAQ, but in manual
(which I tried beforehand). Plot window disappearing too fast -- FAQ
materiel; uint64 initialisation -- not FAQ materiel.

The quoted FAQ section contains info on integers, floating point precision
and symbolic computation, all bunched up together. This is wrong.

Also, talking about integers presupposes talking about *limits, not
precision* -- how big are the integer numbers you can use with the help of
data types: uint32, uint64 etc.

If uint64() works by converting its input somehow through the IEEE floats
(16 positions after the decimal point), than it is doing it wrong, anyway.
1) In my snip, `b` should be assigned the correct value (0.9{repeated 16
times}E+1). 2) The rounding of the input shouldn't go unnoticed, and ideally
should throw a rounding exception or rounding error (we talk integers which
have *absolute precision* by definition).

I've corrected the part on integer limit in FAQ, but the rest of the
comments stands.

Is the FAQ really a place for the ads?







--
View this message in context: 
http://octave.1599824.n4.nabble.com/uint64-typecast-can-t-be-initialised-with-more-than-15-decimal-positions-tp4650508p4650520.html
Sent from the Octave - General mailing list archive at Nabble.com.