Re: outdated m4sugar

[Eric Blake]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

[adding m4-discuss]

According to Joel E. Denny on 7/11/2008 3:00 AM:
| Hi Eric,
|
|
| Patching Bison so that it works with either LIFO or FIFO m4wrap is
| straight-forward.  I just removed the code in m4_wrap in m4_init and
| rewrote Bison to invoke it after processing the skeletons.  Only one use
| of m4wrap remains, and so order no longer matters.  Bison's make check
| then passes for m4 1.4.6 or m4 built from today's git sources.

It sounds like you built the master branch (will become m4 2.0) rather
than branch-1.6 (will become 1.6).  There are even more changes on the
master branch that might need attention, but that branch is not as stable,
so I'm not as worried about it at the moment, when compared with 1.6.

|
| For the new m4, I had to update m4sugar.m4 for the rename of symbols to
| m4symbols.  I haven't figured out why, but this change doesn't hurt with
| m4 1.4.6.

m4symbols is a new addition on the 2.0 branch.  It was previously named
symbols (if you use the alpha-quality 1.4o release), but the introduction
of a new builtin macro that matches an English word and has an effect when
used without arguments was ill-advised, so we renamed it to m4symbols.  I
haven't decided yet whether I should add it to the 1.6 branch (after all,
1.6 is a version bump, making it an opportune time to add some useful
features, even though it certainly won't have all the bells and whistles
such as dynamic modules from the 2.0 release).

|
| I also had to make Bison pass -g to m4 just in case POSIXLY_CORRECT is
| set.  This does not work with m4 1.4.6, but Bison could just unset
| POSIXLY_CORRECT when invoking m4 if we need to support m4 1.4.6 for some
| reason.

For now, -g is also not implemented in m4 1.6.  I'm opposed to having to
make bison unset POSIXLY_CORRECT - the (masochistic) users who set it do
so for a reason, and we shouldn't second-guess them (in particular,
consider what happens if the user wants POSIXLY_CORRECT to affect the
grandchild processes run via syscmd).  If POSIXLY_CORRECT changes m4
behavior, then it should be the use of a non-POSIX option (-g) to override
that and state that GNU extensions are explicitly allowed, and NOT the
unsetting of the environment variable.

On the master branch, I've been trying to reduce the number of places
where POSIXLY_CORRECT affects m4 behavior, as well as making the effects
triggered by POSIXLY_CORRECT independently selectable.  But you should
also note that with 1.4.x and with the current branch-1.6 (unless I add
- -g), POSIXLY_CORRECT currently does not change behavior.  In other words,
m4sugar should work without having to unset POSIXLY_CORRECT, and the rule
merely becomes that if -g is available, you use it.  Unfortunately, this
means that if bison is built against 1.4.x/1.6, but then the user upgrades
to 2.0, then a compile-time check for the existence of 'm4 -g' will have
failed and the upgraded m4 will then fail if POSIXLY_CORRECT is set; on
the other hand, I don't want the expense of a runtime check in every bison
run to find out whether m4 supports -g.

Maybe I should just add -g as a no-op to 1.6, since POSIXLY_CORRECT is a
no-op there.  Then, we do the following release scheduling, so that at all
times, the latest stable release of bison and m4 are compatible:

bison 2.4 - does not use -g, always works with m4 1.4.x and 1.6, but only
works with 2.0 when POSIXLY_CORRECT is not set, behind-the-scene upgrade
from m4 1.4.x to 1.6 is safe but not from 1.6 to 2.0

m4 1.6 - works with bison 2.4 and newer

bison xxx - requires at least m4 1.6 and always works with 2.0, blindly
uses -g (even when it is a no-op), behind-the-scene upgrade from m4 1.6 to
2.0 is safe

m4 2.0 - works with bison xxx and newer

|
| I can apply a patch later if this is the way we want to go.
|
| Btw, HACKING in m4's latest git sources says it depends on Automake
| 1.10.1, which does not appear to provide the pkglibexecdir feature you
| requested a couple of years ago.  That stumped me for a while.

branch-1.6 does not depend on pkglibexecdir.  But the master branch does;
and looking at Automake NEWS, you are correct that pkglibexecdir is only
in 1.10a.  I'll patch m4 shortly to fix this dependency (another reason
that m4 2.0 is unstable - it currently depends on too many bleeding edge
unreleased packages).

|
|> At any rate, I'm
|> debating about making bison a show-stopper for the release of m4 1.6
(ie. I'm
|> reluctant to release m4 1.6 today, since it would mean the latest released
|> versions of bison and m4 would be incompatible).
|
| If I just patch Bison as I described above and then push Bison 2.4 out,
| you could then release m4 1.6.

Don't worry about making Bison 2.4 gracefully deal with m4 2.0; right now,
I'm only concerned with the release of m4 1.6.  We can save m4 2.0 support
for the future (autoconf will probably also need some tweaks to work with
m4 2.0 features).

|  However, I had hoped to wait longer before
| 2.4 for several reasons: more user feedback on 2.3b, documenting XML
| support, and fixing some internationalization problems (the error messages
| we moved/added to the skeletons are the most difficult problem).  Maybe
| these things (except the recent internationalization bug report) will just
| have to wait for 2.5.

There's still several things I want in m4 1.6; so I'm guessing at least
another month before I'm ready to go.

|
|> Also, I'm considering making m4 1.6 treat the -d command-line option in a
|> position-dependent manner.
|
| Sure, we can fix that too.  But why not give it a different name in order
| not to break backward compatibility?  Just a thought.  I'm not too
| concerned for Bison here because --trace is undocumented for users.

I'll have to think about it.  I already made the order-dependent -d change
on the master branch; right now I'm in the middle of debating whether to
port that to branch-1.6.  The reason I made it on the master branch was
because I added the ability to make frozen files track the current
- -d/debugmode state, and making -d order-dependent was the only way to make
debugmode interact sanely with frozen files (basically, if both -d and the
frozen file in -F specify different debug modes, determining which one
takes precedence is easier to document if -d is order-dependent).  The
master branch also supports usage such as 'm4 -d+f' (similar to the 1.4.x
m4_debugmode([+f])), so that the command line can undo debugmode settings
made in earlier files.  By the way, I'm also considering porting some of
the frozen file improvements from master to branch-1.6 (the ability to
freeze the state of which macros are being traced will make autom4te's
notion of pre-traced macros easier to implement, compared to the current
1.4.x solution of having to update autoconf releases every time a third
party package wants to enlarge the set of pre-traced macros.)

|
|> Meanwhile, bison uses a check for m4 1.3's ability to use frozen files
|> (m4/m4.m4, also borrowed from autoconf), even though bison itself does not
|> appear to use frozen files.  I don't know how well the current BSD m4
would
|> fare with bison's m4 usage patterns
|
| Bison's README says Bison requires GNU M4.  (But I've been meaning to
| update it to say 1.4.6 as we've been saying in release announcements.)

What does 1.4.6 provide that 1.4.5 doesn't?  In other words, why the
discrepency between what autoconf requires and what bison requires, and
should autoconf be similarly bumped to 1.4.6?  And let's fix m4/m4.m4 to
actually require 1.4.6, rather than its (outdated) test that lets m4 1.3
be selected.

|
|> I am willing to file copyright papers and help in the effort of updating
|> bison's usage of m4sugar, if you would like, since I maintain GNU m4
and made
|> the bulk of the autoconf m4sugar improvements over the past 3 years.
|
| Thanks!  Actually, we probably need to do copyright papers for you anyway
| because of your previous patch.

My previous patch falls under the trivial rules, but I'm already starting
the paperwork process.

|
| However, of the issues you've mentioned, updating m4sugar seems like the
| least pressing.  We can handle it gradually if it turns out to be
| difficult.

- --
Don't work too hard, make some time for fun as well!

Eric Blake             address@hidden
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Cygwin)
Comment: Public key at home.comcast.net/~ericblake/eblake.gpg
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAkh3XJwACgkQ84KuGfSFAYC8qgCgvbjUzHetHRWktNRdTtAYXEQQ
UR4AniFaAA9WrwZLBjhuP260QtaFWqA1
=P9XN
-----END PGP SIGNATURE-----