Re: totalorder*: Fix test failures on PA-RISC and MIPS CPUs

[Bruno Haible]

On Linux/sh4 (configuring again with
  gl_cv_func_totalorder_in_libm=no gl_cv_func_totalorder_no_libm=no \
  gl_cv_func_totalorderf_in_libm=no gl_cv_func_totalorderf_no_libm=no \
  gl_cv_func_totalorderl_in_libm=no gl_cv_func_totalorderl_no_libm=no \
  ./configure
), I see the same test failures:

FAIL: test-totalorder
=====================

Failed: i=0 j=1
Failed: i=1 j=0
Failed: i=12 j=13
Failed: i=13 j=12
FAIL test-totalorder (exit status: 1)

FAIL: test-totalorderf
======================

Failed: i=0 j=1
Failed: i=1 j=0
Failed: i=12 j=13
Failed: i=13 j=12
FAIL test-totalorderf (exit status: 1)

FAIL: test-totalorderl
======================

Failed: i=0 j=1
Failed: i=1 j=0
Failed: i=12 j=13
Failed: i=13 j=12
FAIL test-totalorderl (exit status: 1)

The reason is that the SH4 CPU defines the distinction between quiet and
signalling NaNs like hppa and mips. See the "SH-4 Software Manual"
<https://www.renesas.com/us/en/document/mas/sh-4-software-manual> page 143.

This patch fixes it.


2023-10-14  Bruno Haible  <bruno@clisp.org>

        totalorder*: Fix test failures on SH4.
        * lib/snan.h: Update comment.
        * lib/totalorderf.c (totalorderf): On sh4, invert bit 22 before
        comparing two NaNs.
        * lib/totalorder.c (totalorder): On sh4, invert bit 51 before comparing
        two NaNs.
        * lib/totalorderl.c (totalorderl): On sh4, invert bit 51 or 47 of the
        xhi, yhi parts before comparing two NaNs.

diff --git a/lib/snan.h b/lib/snan.h
index ebf88ab0c5..abd95318d7 100644
--- a/lib/snan.h
+++ b/lib/snan.h
@@ -32,7 +32,7 @@
    This bit is
      *  == 0 to indicate a quiet NaN or Infinity,
         == 1 to indicate a signalling NaN,
-        on these CPUs: hppa, mips.
+        on these CPUs: hppa, mips, sh4.
      *  == 1 to indicate a quiet NaN,
         == 0 to indicate a signalling NaN or Infinity,
         on all other CPUs.
diff --git a/lib/totalorder.c b/lib/totalorder.c
index edbcd6998b..ee90ebb1f9 100644
--- a/lib/totalorder.c
+++ b/lib/totalorder.c
@@ -46,7 +46,7 @@ totalorder (double const *x, double const *y)
   /* At this point, *X and *Y are NaNs with the same sign bit.  */
 
   unsigned long long extended_sign = -!!xs;
-#if defined __hppa || defined __mips__
+#if defined __hppa || defined __mips__ || defined __sh__
   /* Invert the most significant bit of the mantissa field.  Cf. snan.h.  */
   extended_sign ^= (1ULL << 51);
 #endif
diff --git a/lib/totalorderf.c b/lib/totalorderf.c
index 4cbfa06ffd..fa2a1245de 100644
--- a/lib/totalorderf.c
+++ b/lib/totalorderf.c
@@ -46,7 +46,7 @@ totalorderf (float const *x, float const *y)
   /* At this point, *X and *Y are NaNs with the same sign bit.  */
 
   unsigned int extended_sign = -!!xs;
-#if defined __hppa || defined __mips__
+#if defined __hppa || defined __mips__ || defined __sh__
   /* Invert the most significant bit of the mantissa field.  Cf. snan.h.  */
   extended_sign ^= (1U << 22);
 #endif
diff --git a/lib/totalorderl.c b/lib/totalorderl.c
index 6982762a0f..a06e70d55f 100644
--- a/lib/totalorderl.c
+++ b/lib/totalorderl.c
@@ -55,7 +55,7 @@ totalorderl (long double const *x, long double const *y)
 
   if (sizeof (long double) <= sizeof (unsigned long long))
     {
-#if defined __hppa || defined __mips__
+#if defined __hppa || defined __mips__ || defined __sh__
       /* Invert the most significant bit of the mantissa field.  Cf. snan.h.  
*/
       extended_sign ^= (1ULL << 51);
 #endif
@@ -67,7 +67,7 @@ totalorderl (long double const *x, long double const *y)
     }
 
   unsigned long long extended_sign_hi = extended_sign;
-#if defined __hppa || defined __mips__
+#if defined __hppa || defined __mips__ || defined __sh__
   /* Invert the most significant bit of the mantissa field.  Cf. snan.h.  */
   extended_sign_hi ^=
     (1ULL << (LDBL_MANT_DIG == 106