Re: address@hidden: Bug#94207: fmt dumps core with large -w]

[Jim Meyering]

Jim Meyering <address@hidden> wrote:

> Herbert Xu <address@hidden> wrote:
> ...
> > (address@hidden) /tmp % fmt -w 100000 < /etc/fstab
>> [snip output]
>> zsh: segmentation fault (core dumped)  fmt -w 100000 < /etc/fstab
>
> Thanks for the report.
>
> Here's a patch:
>
>       The command `echo foo| fmt -w 100000' would cause fmt to segfault.
>
>       * src/fmt.c [struct Word]: Declare members length, space, and
>       line_length to be of type `int', not short.
>       (MAXCOST): Define using TYPE_MAXIMUM.
>       Reported by Herbert Xu.

I found that the above wasn't enough.
The following patch avoids the segfault even when a maximum
width of 2^31-1 (INT_MAX) is chosen.

However, there remains the problem you pointed out, regarding
the COST metric.  But that should arise only with a very large
maximum line length, *and* with lines of length about sqrt(INT_MAX/10).

Luckily, not many people worry about the looks of paragraphs
that are formatted to have lines 14,000+-bytes long.


        The command `echo a|./fmt -2147483647' would cause fmt to segfault.
        * src/fmt.c (fmt_paragraph): Test for sentinal directly, rather than
        doing arithmetic with it's potentially large (INT_MAX) length.

Index: fmt.c
===================================================================
RCS file: /fetish/textutils/src/fmt.c,v
retrieving revision 1.66
diff -u -p -u -p -r1.66 fmt.c
--- fmt.c       2001/09/12 06:41:01     1.66
+++ fmt.c       2001/09/12 08:38:56
@@ -816,6 +816,14 @@ fmt_paragraph (void)
              start->next_break = w;
              start->line_length = len;
            }
+
+         /* This is a kludge to keep us from computing `len' as the
+            sum of the sentinel length and some non-zero number.
+            Since the sentinel w->length may be INT_MAX, adding
+            to that would give a negative result.  */
+         if (w == word_limit)
+           break;
+
          len += (w - 1)->space + w->length;    /* w > start >= word */
        }
       while (len < max_width);