WORDS_BIGENDIAN not defined on sparc Solaris 10!

[Jim Meyering]

This is a nasty bug.

I noticed failures of coreutils' md5 and sha*-related tests
on sparc/Solaris systems.  It came down to the fact that
config.hin does the following (in config.h this bit is identical
on the system in question):

/* Define WORDS_BIGENDIAN to 1 if your processor stores words with the most
   significant byte first (like Motorola and SPARC, unlike Intel and VAX). */
#if defined __BIG_ENDIAN__
# define WORDS_BIGENDIAN 1
#elif ! defined __LITTLE_ENDIAN__
/* #undef WORDS_BIGENDIAN */
#endif

And since __BIG_ENDIAN__ is not defined (neither with
recent gcc nor with sunc99), WORDS_BIGENDIAN is not defined.
And lib/md5.c is incorrect on that processor without the definition.

This code has been around for 9(!) months, now.
It was introduced here:

2007-02-27  Paul Eggert  <address@hidden>

        * NEWS: AC_C_BIGENDIAN now supports universal binaries a la Mac OS X.
        * doc/autoconf.texi (C Compiler): Document this.  There is a new
        extra argument ACTION-IF-UNIVERSAL.
        * lib/autoconf/c.m4 (AC_C_BIGENDIAN): Implement this.
        Add support for Solaris-style _LITTLE_ENDIAN and _BIG_ENDIAN.
        Reindent for sanity's sake.

The problem is that when no __*ENDIAN*__ symbol is defined, configure
performs its run test and then must use that result unconditionally.
So, for now, I propose to remove that AH_VERBATIM block, like this:

Using autoconf with the patch below, coreutils now passes its tests
on big-endian systems like sparc.  If no one objects, I'll commit
this on Thursday (but with changes to NEWS too).

=============================================================

        Don't hide the definition of WORDS_BIGENDIAN!
        * lib/autoconf/c.m4 (AC_C_BIGENDIAN): Remove AH_VERBATIM whose
        "#if defined __BIG_ENDIAN__" would effectively remove the definition
        of WORDS_BIGENDIAN that is required on e.g., sparc-based systems.

diff --git a/lib/autoconf/c.m4 b/lib/autoconf/c.m4
index b67de57..1b78f4e 100644
--- a/lib/autoconf/c.m4
+++ b/lib/autoconf/c.m4
@@ -1395,15 +1395,7 @@ fi
 #                 [ACTION-IF-UNIVERSAL])
 # -------------------------------------------------------------------------
 AC_DEFUN([AC_C_BIGENDIAN],
-[AH_VERBATIM([WORDS_BIGENDIAN],
-[/* Define WORDS_BIGENDIAN to 1 if your processor stores words with the most
-   significant byte first (like Motorola and SPARC, unlike Intel and VAX). */
-#if defined __BIG_ENDIAN__
-# define WORDS_BIGENDIAN 1
-#elif ! defined __LITTLE_ENDIAN__
-/* #undef WORDS_BIGENDIAN */
-#endif])dnl
- AC_CACHE_CHECK([whether byte ordering is bigendian], [ac_cv_c_bigendian],
+[AC_CACHE_CHECK([whether byte ordering is bigendian], [ac_cv_c_bigendian],
    [ac_cv_c_bigendian=unknown
     m4_ifval(m4_ifdef([AH_HEADER], 1)[$4],
       [# See if __BIG_ENDIAN__ or __LITTLE_ENDIAN__ is defined.
@@ -1515,7 +1507,10 @@ AC_DEFUN([AC_C_BIGENDIAN],
  case $ac_cv_c_bigendian in #(
    yes)
      m4_default([$1],
-       [AC_DEFINE([WORDS_BIGENDIAN], 1)]);; #(
+       [AC_DEFINE([WORDS_BIGENDIAN], 1,
+        [Define WORDS_BIGENDIAN to 1 if your processor stores words with the 
most
+         significant byte first (like Motorola and SPARC, unlike Intel and 
VAX).]
+       )]);; #(
    no)
      $2 ;; #(
    universal)
--
1.5.3.6.950.g92b7b