Re: gawk compl()

[Andreas Schwab]

Stepan Kasal <address@hidden> writes:

> Hello,
>
> On Mon, Oct 10, 2005 at 09:29:55AM -0400, Andrew J. Schorr wrote:
>> On Mon, Oct 10, 2005 at 02:19:49PM +0200, Karel Zak wrote:
>> > $ gawk 'BEGIN {printf("%#x\n", compl(0xFFFF0000)) }'
>> > 0x1fffff0000ffff
> ...
>> So, in fact, the calculation is being done with 64 bits.
>> Therefore ~0xFFFF0000 = 0xFFFFFFFF0000FFFF.  This is
>> then converted to floating point and gets truncated
>> in the process.
>
> I have to admit I still don't understand how 0xffffffff0000ffff
> becomes 0x1fffff0000ffff.

        /*
         * If uintmax_t is so wide that AWKNUM cannot represent all its
         * values, strip leading nonzero bits of integers that are so large
         * that they cannot be represented exactly as AWKNUMs, so that their
         * low order bits are represented exactly, without rounding errors.
         * This is more desirable in practice, since it means the user sees
         * integers that are the same width as the AWKNUM fractions.
         */
        if (AWKNUM_FRACTION_BITS < CHAR_BIT * sizeof n)
                n &= ((uintmax_t) 1 << AWKNUM_FRACTION_BITS) - 1;

Andreas.

-- 
Andreas Schwab, SuSE Labs, address@hidden
SuSE Linux Products GmbH, Maxfeldstraße 5, 90409 Nürnberg, Germany
Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."