Re: UBSAN error in lib/sh/random.c:79

[Andreas Schwab]

On Jan 07 2023, Greg Wooledge wrote:

> I think this patch might be correct:
>
>
> --- lib/sh/random.c.orig      2023-01-07 12:26:09.049950519 -0500
> +++ lib/sh/random.c   2023-01-07 12:26:27.469974730 -0500
> @@ -70,8 +70,8 @@
>       There are lots of other combinations of constants to use; look at
>       
> https://www.gnu.org/software/gsl/manual/html_node/Other-random-number-generators.html#Other-random-number-generators
>  */
>  
> -  bits32_t h, l, t;
> -  u_bits32_t ret;
> +  bits32_t t;
> +  u_bits32_t h, l, ret;
>  
>    /* Can't seed with 0. */
>    ret = (last == 0) ? 123459876 : last;
>
>
> I tested it briefly, and it builds cleanly and produces the same random
> results as the unpatched version, at least on my system (compiled with
> gcc 10.2.1).

The assignment t = 16807 * l - 2836 * h can still overflow, because if l
and h are unsigned, the computed value can never be negative, but it
becomes bigger than INT_MAX if 2836 * h is bigger than 16807 * l (the
unsigned result is computed modulo UINT_MAX+1).

I think the original overflow can only happen if the argument of
intrand32 is bigger than INT_MAX.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510  2552 DF73 E780 A9DA AEC1
"And now for something completely different."