Re: patch to avoid huge buffer in scan-skel.l

[Paul Eggert]

> From: Akim Demaille <address@hidden>
> Date: Thu, 09 Jan 2003 10:08:24 +0100

>  Paul> However, I'd rather not have Bison create unbounded buffers
>  Paul> without good reason.  
> 
> Why?  I mean, we use Flex and we know that it does this properly,

I was prompted to do the change because I observe from the code that
it does not always reallocate buffers properly, on some hosts where
__STDC__ is 0 (or not defined), __cplusplus is not defined, the user
does not define YY_MALLOC_DECL, and size_t is not the same width as
unsigned int.  This is because it passes an unsigned int value to an
unprototyped malloc.

My theory is that this might cause a problem on older Alpha hosts, if
a buffer gets reallocated.  I can't check the theory, but I thought
it better to be conservative.

The "right" fix, of course, is to fix flex so that it doesn't generate
dubious code, but I didn't have the time to do that, and anyway even
if I fixed my own copy the next maintainer will still generate a buggy
scanner when he re-runs flex.

> unboundness here would refer to an arbitrary long line, which is
> often considered not to be the typical case.

True: sorry, I misread the code, so the ChangeLog entry isn't
quite accurate.

> Fetching char per char is quite known to cost in Lex scanners.  This
> is taken from the documentation of Flex (but I agree anyway that M4
> dominates the costs today):

Perhaps when we fix the m4 problem we can revisit the issue.  Frankly
I have grown somewhat disillusioned with m4 for this application,
given all the quoting problems I've had to deal with in the last
couple of months.