[PATCH] sed: avoid data loss when an input line is 2^31 bytes or longer

[Jim Meyering]

I was surprised to see that sed -i could nuke its input.
Even sed without -i misbehaves:

    $ perl -le 'print "v"x(2**31-1)' |sed 's/^/v/' |wc -c
    0

Here's the fix:

>From 81ce070727b225a1e23e5a48f775811c8a9e7366 Mon Sep 17 00:00:00 2001
From: Jim Meyering <address@hidden>
Date: Mon, 11 Jul 2011 17:22:26 +0200
Subject: [PATCH] avoid silent data loss when an input line is 2^31 bytes or
 longer

If one line in a file has length (including newline) of 2^31, then
applying sed -i to that file truncates it to size 0.  I first
noticed it like this: Create a file with line of length 2^31-1
  $ perl -le 'print "v"x(2**31-1)' > k
Then prepend a byte to that line:
  $ sed -i 's/^/v/' k
Surprise!  The file is empty.
* sed/utils.c (ck_getline): Declare "result" to be of type ssize_t,
rather than int, to match the return type of getline.
---
 ChangeLog   |   13 +++++++++++++
 NEWS        |    2 ++
 sed/utils.c |    4 ++--
 3 files changed, 17 insertions(+), 2 deletions(-)

diff --git a/ChangeLog b/ChangeLog
index 59f4a64..c38beb5 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,16 @@
+2011-07-11  Jim Meyering  <address@hidden>
+
+       avoid silent data loss when an input line is 2^31 bytes or longer
+       If one line in a file has length (including newline) of 2^31, then
+       applying sed -i to that file truncates it to size 0.  I first
+       noticed it like this: Create a file with line of length 2^31-1
+         $ perl -le 'print "v"x(2**31-1)' > k
+       Then prepend a byte to that line:
+         $ sed -i 's/^/v/' k
+       Surprise!  The file is empty.
+       * sed/utils.c (ck_getline): Declare "result" to be of type ssize_t,
+       rather than int, to match the return type of getline.
+
 2011-07-14  Padraig Brady  <address@hidden>

        * execute.c (open_next_file): Only reopen stdin on Windows/DOS.
diff --git a/NEWS b/NEWS
index 89ea757..7b56799 100644
--- a/NEWS
+++ b/NEWS
@@ -1,5 +1,7 @@
 Sed 4.2.2

+* don't misbehave (truncate input) for lines of length 2^31 and longer
+
 * fix endless loop on incomplete multibyte sequences

 * -u also does unbuffered input, rather than unbuffered output only
diff --git a/sed/utils.c b/sed/utils.c
index 5b5066b..7a00f67 100644
--- a/sed/utils.c
+++ b/sed/utils.c
@@ -1,5 +1,5 @@
 /*  Functions from hack's utils library.
-    Copyright (C) 1989, 1990, 1991, 1998, 1999, 2003, 2008, 2009
+    Copyright (C) 1989, 1990, 1991, 1998, 1999, 2003, 2008, 2009, 2011
     Free Software Foundation, Inc.

     This program is free software; you can redistribute it and/or modify
@@ -264,7 +264,7 @@ ck_getline(text, buflen, stream)
   size_t *buflen;
   FILE *stream;
 {
-  int result;
+  ssize_t result;
   bool error;

   error = ferror (stream);
--
1.7.6.430.g34be2