coreutils
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[INSTALLED 5/5] sort: fix unlikely int overflow with -r


From: Paul Eggert
Subject: [INSTALLED 5/5] sort: fix unlikely int overflow with -r
Date: Tue, 17 May 2022 19:32:25 -0700

* src/sort.c (keycompare, compare): Don’t overflow if -r is
specified and a comparison function returns INT_MIN, as this
causes the comparison to have undefined behavior (typically the
reverse of correct).  glibc memcmp on s390x reportedly returns
INT_MIN in some cases, so this is not a purely academic issue.
---
 src/sort.c | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)

diff --git a/src/sort.c b/src/sort.c
index 29c9f39f3..81045b1ab 100644
--- a/src/sort.c
+++ b/src/sort.c
@@ -2794,7 +2794,9 @@ keycompare (struct line const *a, struct line const *b)
         }
     }
 
-  return key->reverse ? -diff : diff;
+  if (key->reverse)
+    diff = diff < 0 ? -1 : -diff;
+  return diff;
 }
 
 /* Compare two lines A and B, returning negative, zero, or positive
@@ -2839,7 +2841,9 @@ compare (struct line const *a, struct line const *b)
         diff = (alen > blen) - (alen < blen);
     }
 
-  return reverse ? -diff : diff;
+  if (reverse)
+    diff = diff < 0 ? 1 : -diff;
+  return diff;
 }
 
 /* Write LINE to output stream FP; the output file's name is
-- 
2.36.1




reply via email to

[Prev in Thread] Current Thread [Next in Thread]