Re: Undefined behavior in conv-integer.i.c?

[Mark H Weaver]

Mark H Weaver <address@hidden> writes:

> Miroslav Lichvar <address@hidden> writes:
>
>> I was looking at a problem with guile-1.8.8 when compiled with
>> gcc-6.0. Two of the tests from the test suite were failing with
>> strange "out of range" errors [1]. After some investigation I think
>> the bug is that the code in libguile/conv-integer.i.c relies on
>> overflow of signed integers in the following code (starting on line
>> 77), specifically -TYPE_MIN being less than zero. Adding -fwrapv to
>> CFLAGS worked as a workaround for me.
>>
>>           if (mpz_sgn (SCM_I_BIG_MPZ (val)) >= 0)
>>             {
>>               if (n < 0)
>>                 goto out_of_range;
>>             }
>>           else
>>             {
>>               n = -n;
>>               if (n >= 0)
>>                 goto out_of_range;
>>             }
>
> Thanks for bringing this to our attention.  I've attached a preliminary
> patch to address these issues on the 'stable-2.0' branch.
>
>> Looking at the current guile code, conv-integer.i.c is identical to
>> what it was in 1.8.8, but interestingly the tests didn't fail for me.
>> Maybe something else is preventing gcc from using the optimization?
>
> The build system of recent Guile 2.0.x automatically adds -fwrapv to
> CFLAGS where supported.  However, I hope to remove -fwrapv in the
> future, when we gain confidence that no code in Guile depends on it.
>
>> I'm not sure what would be the best way to fix it. Maybe n should
>> really be unsigned and compared to the maximum values, but what would
>> be the absolute value of TYPE_MIN if it should work also with other
>> integer representations than two's complement?
>
> My approach was to compare (abs_n - 1) to -(TYPE_MIN + 1) in the case
> where n is negative.

I pushed these fixes as commit 4b60562820d001674ec7124c4a10391ecf7e44c3
to the stable-2.0 branch, which will become guile-2.0.12.

     Thanks,
       Mark