qemu-devel
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [PATCH] utils: Reduce chance of rounding inaccuracy in qemu_strtosz.


From: Richard Henderson
Subject: Re: [PATCH] utils: Reduce chance of rounding inaccuracy in qemu_strtosz.
Date: Sat, 13 Mar 2021 15:48:43 -0600
User-agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.7.1

On 3/11/21 2:07 PM, Eric Blake wrote:
+        /*
+         * Add in a fudge-factor (2^53 when double is IEEE format) for
+         * all scales less than P (2^50), so that things like
+         * 12.345M with unit 1000 produce 12345000 instead of
+         * 12344999.
+         */
+        if (mul > 1e49) {

The comment says less than, the code says greater than.


An alternative patch might be writing (uint64_t)(fraction * mul + 0.5)
(that is, introduce the fudge factor after the multiplication instead
of before).  Preferences?

I think I would prefer this, or for further rounding error reduction, fma(fraction, mul, 0.5).


r~



reply via email to

[Prev in Thread] Current Thread [Next in Thread]