[avr-libc-dev] Bug in math library

[Colin O'Flynn]

Hello,

I have found a bug in the math library, which is responsible for the 0/0 
error. It comes down to this:

In various functions there is a test that checks if the number is 0. This is 
used in the division routine, among others. A quick grep for the TST macro:

address@hidden fplib]$ grep TST *.*
addsf3x.S:    TST     rB3                ;
atan2.S:        TST     x3              ; x == 0?
atan2.S:        TST     y3              ; y == 0?
atan2.S:        TST     y3              ; y == 0?
atan2.S:        TST     x3              ; x == 0?
atan.S: TST     magn                    ; |x| > 1 ?
ceil.S:    TST     rA3               ;                        \ 0.0  f�r X <= 
0.0
ceil.S:    TST     rAE        ; any bit shiftet out? if LSB > 1 then rAE is 
zerodivsf3x.S:    TST     rB3
divsf3x.S:    TST     rA3
exp.S:   TST   rA3
floor.S:    TST     rAE
fmod.S:    TST     rB3          ; -0 ?
fmod.S:    TST     rB2
fp_merge.S:    TST     rA3            ; exponent == 0
fp_merge.S:    TST     rA2
fp_merge.S:    TST   rT0
fp_powerseries.S:    TST    rGrade
fp_powerseries.S:    TST    rA3                 ; exponent == 0
fp_powerseries.S:    TST    rA2
frexp.S:        TST     rA3
ldexp.S:        TST     rA3                     ;
ldexp.S:        TST     rPH
log.S:   TST   rA3
modf.S:        TST     rA3              ; if zero return zero for *int and 
mantissa
mulsf3x.S:    TST     rA3
mulsf3x.S:    TST     rB3
mulsf3x.S:    TST     loop
mulsf3x.S:    TST     rA2
pow.S:   TST   rB3
pow.S:   TST   rA3
sqrt.S:    TST     rA3
strtod.S:       TST     rA3
strtod.S:       TST    rExp
strtod.S:       TST    rB3
strtod.S:       TST     rA3
address@hidden fplib]$

The TST macro is an AND of the register with itself, which if the result is 
zero will set the Z flag, so a BREQ or whatnot can be used after it. In all 
cases it is checking if the exponent is zero - I assume if that is the case 
than the floating point number is zero?

At any rate the problem lies in the fp_split.S file. The file says this:

CLR     rBE            ; clear fraction extension
cp      rBE,rB3        ; set C if nonzero exponent.  -MM 2000-11-25
brcs    1f
ldi     rB3,1          ; zero exponent really means 1 with hidden bit = 0

If the exponent is nonzero, it sets it to 1. This breaks the test completely - 
as far as I can tell this is a pretty big problem. I don't know enough about 
avr-libc's "special" floating-point format to understand why there is this 
'hidden bit' thought, which is why I just didn't make a patch.

At any rate it is my guess that the math library has been broken since 
2000-11-15, which is when it looks like that fragment was added. To me this 
seems fairly large, as it also breaks a number of other routines with "zero 
tests" in them.

So whats going on there, can anyone fill me in on the details/create a patch? 
I could be totally off-bar here, but it looks to me like an important bug.

Warm Regards,

 -Colin