Re: %prec using undefined token

[Joel E. Denny]

On Wed, 30 Dec 2009, Joel E. Denny wrote:

> The patch below restores the complaint (as an error not a warning) for the 
> case when a %prec symbol is not defined, but it maintains the complaint 
> for the case when the symbol is a nonterminal.  This patch might break 
> compatibility with Bison 2.4 and 2.4.1.  I'm trying to decide if I should 
> rewrite it to be a warning in Bison 2.4.2 but an error in Bison 2.5 and 
> later.

I've decided I should, and I've found an easy way to do it.  I'm planning 
to push the first patch below to branch-2.4.2.  I'm planning to push both 
patches below to branch-2.5 and master.  I'll wait a little for comments.

>From 293185465c93e0e4818ea738b655dd9bace49a7c Mon Sep 17 00:00:00 2001
From: Joel E. Denny <address@hidden>
Date: Wed, 30 Dec 2009 03:20:11 -0500
Subject: [PATCH] POSIX: warn if %prec's token was not defined.

Reported by Florian Krohm at
<http://lists.gnu.org/archive/html/bug-bison/2009-12/msg00005.html>.
* NEWS (2.4.2): Document.
* src/reader.c (grammar_rule_check): Implement.
(grammar_current_rule_prec_set): Add comments explaining that we
here assume a %prec identifier is a token, but we still manage
to support POSIX.
* tests/input.at (%prec's token must be defined): New test
group.
---
 ChangeLog      |   13 +++++++++++++
 NEWS           |   10 ++++++++++
 src/reader.c   |   19 +++++++++++++++++++
 tests/input.at |   20 ++++++++++++++++++++
 4 files changed, 62 insertions(+), 0 deletions(-)

diff --git a/ChangeLog b/ChangeLog
index f32a46e..b65f8c7 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,16 @@
+2009-12-30  Joel E. Denny  <address@hidden>
+
+       POSIX: warn if %prec's token was not defined.
+       Reported by Florian Krohm at
+       <http://lists.gnu.org/archive/html/bug-bison/2009-12/msg00005.html>.
+       * NEWS (2.4.2): Document.
+       * src/reader.c (grammar_rule_check): Implement.
+       (grammar_current_rule_prec_set): Add comments explaining that we
+       here assume a %prec identifier is a token, but we still manage
+       to support POSIX.
+       * tests/input.at (%prec's token must be defined): New test
+       group.
+
 2009-12-21  Joel E. Denny  <address@hidden>
 
        YYFAIL: deprecate.
diff --git a/NEWS b/NEWS
index 9ef543e..9045431 100644
--- a/NEWS
+++ b/NEWS
@@ -3,6 +3,16 @@ Bison News
 
 * Changes in version 2.4.2 (????-??-??):
 
+** `%prec IDENTIFIER' requires IDENTIFIER to be defined separately.
+
+  POSIX specifies that an error be reported for any identifier that does
+  not appear on the LHS of a grammar rule and that is not defined by
+  %token, %left, %right, or %nonassoc.  Bison 2.3b and later lost this
+  error report for the case when an identifier appears only after a
+  %prec directive.  It is now restored.  However, for backward
+  compatibility with recent Bison releases, it is only a warning for
+  now.  In Bison 2.5 and later, it will return to being an error.
+
 ** Detection of GNU M4 1.4.6 or newer during configure is improved.
 
 ** Warnings from gcc's -Wundef option about undefined YYENABLE_NLS,
diff --git a/src/reader.c b/src/reader.c
index 74d7dca..38191ee 100644
--- a/src/reader.c
+++ b/src/reader.c
@@ -292,6 +292,15 @@ grammar_rule_check (const symbol_list *r)
            warn_at (r->location, _("unset value: $$"));
        }
   }
+
+  /* See comments in grammar_current_rule_prec_set for how POSIX
+     mandates this complaint.  It's only for identifiers, so skip
+     it for char literals and strings, which are always tokens.  */
+  if (r->ruleprec
+      && r->ruleprec->tag[0] != '\'' && r->ruleprec->tag[0] != '"'
+      && !r->ruleprec->declared && !r->ruleprec->prec)
+    warn_at (r->location, _("token for %%prec is not defined: %s"),
+             r->ruleprec->tag);
 }
 
 
@@ -364,6 +373,16 @@ grammar_midrule_action (void)
 void
 grammar_current_rule_prec_set (symbol *precsym, location loc)
 {
+  /* POSIX says that any identifier is a nonterminal if it does not
+     appear on the LHS of a grammar rule and is not defined by %token
+     or by one of the directives that assigns precedence to a token.  We
+     ignore this here because the only kind of identifier that POSIX
+     allows to follow a %prec is a token and because assuming it's a
+     token now can produce more logical error messages.  Nevertheless,
+     grammar_rule_check does obey what we believe is the real intent of
+     POSIX here: that an error be reported for any identifier that
+     appears after %prec but that is not defined separately as a
+     token.  */
   symbol_class_set (precsym, token_sym, loc, false);
   if (current_rule->ruleprec)
     complain_at (loc, _("only one %s allowed per rule"), "%prec");
diff --git a/tests/input.at b/tests/input.at
index be84f9c..5cec25b 100644
--- a/tests/input.at
+++ b/tests/input.at
@@ -702,6 +702,26 @@ AT_BISON_CHECK([input.y], [1], [],
 AT_CLEANUP
 
 
+## ------------------------------- ##
+## %prec's token must be defined.  ##
+## ------------------------------- ##
+
+AT_SETUP([[%prec's token must be defined]])
+
+# According to POSIX, a %prec token must be defined separately.
+
+AT_DATA([[input.y]],
+[[%%
+start: %prec PREC ;
+]])
+
+AT_BISON_CHECK([[input.y]], [[0]], [],
+[[input.y:2.8-17: warning: token for %prec is not defined: PREC
+]])
+
+AT_CLEANUP
+
+
 ## -------------------------------- ##
 ## Reject unused %code qualifiers.  ##
 ## -------------------------------- ##
-- 
1.5.4.3

>From 43cddeaa14a63c82b40c6c2ae0bd66f5d9addd39 Mon Sep 17 00:00:00 2001
From: Joel E. Denny <address@hidden>
Date: Wed, 30 Dec 2009 03:14:12 -0500
Subject: [PATCH] POSIX: complain if %prec's token was not defined.

* NEWS (2.5): Document.
* src/reader.c (grammar_rule_check): Convert warning to
complaint.
* tests/input.at (%prec's token must be defined): Update.
---
 ChangeLog      |    8 ++++++++
 NEWS           |    6 ++++++
 src/reader.c   |    4 ++--
 tests/input.at |    4 ++--
 4 files changed, 18 insertions(+), 4 deletions(-)

diff --git a/ChangeLog b/ChangeLog
index faf7b4a..d2111ba 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,5 +1,13 @@
 2009-12-30  Joel E. Denny  <address@hidden>
 
+       POSIX: complain if %prec's token was not defined.
+       * NEWS (2.5): Document.
+       * src/reader.c (grammar_rule_check): Convert warning to
+       complaint.
+       * tests/input.at (%prec's token must be defined): Update.
+
+2009-12-30  Joel E. Denny  <address@hidden>
+
        POSIX: warn if %prec's token was not defined.
        Reported by Florian Krohm at
        <http://lists.gnu.org/archive/html/bug-bison/2009-12/msg00005.html>.
diff --git a/NEWS b/NEWS
index 8e3280a..086325d 100644
--- a/NEWS
+++ b/NEWS
@@ -130,6 +130,12 @@ Bison News
   discussion of how to suppress C preprocessor warnings about YYFAIL
   being unused, see the Bison 2.4.2 NEWS entry.
 
+** `%prec IDENTIFIER' requires IDENTIFIER to be defined separately.
+
+  As promised in Bison 2.4.2's NEWS entry, it is now an error if a token
+  that appears after a %prec directive is not defined by %token, %left,
+  %right, or %nonassoc.  This is required by POSIX.
+
 ** Temporary hack for adding a semicolon to the user action.
 
   Previously, Bison appended a semicolon to every user action for
diff --git a/src/reader.c b/src/reader.c
index 250e4f8..3f00754 100644
--- a/src/reader.c
+++ b/src/reader.c
@@ -327,8 +327,8 @@ grammar_rule_check (const symbol_list *r)
   if (r->ruleprec
       && r->ruleprec->tag[0] != '\'' && r->ruleprec->tag[0] != '"'
       && !r->ruleprec->declared && !r->ruleprec->prec)
-    warn_at (r->location, _("token for %%prec is not defined: %s"),
-             r->ruleprec->tag);
+    complain_at (r->location, _("token for %%prec is not defined: %s"),
+                 r->ruleprec->tag);
 }
 
 
diff --git a/tests/input.at b/tests/input.at
index 3da6137..27f12f2 100644
--- a/tests/input.at
+++ b/tests/input.at
@@ -812,8 +812,8 @@ AT_DATA([[input.y]],
 start: %prec PREC ;
 ]])
 
-AT_BISON_CHECK([[input.y]], [[0]], [],
-[[input.y:2.8-17: warning: token for %prec is not defined: PREC
+AT_BISON_CHECK([[input.y]], [[1]], [],
+[[input.y:2.8-17: token for %prec is not defined: PREC
 ]])
 
 AT_CLEANUP
-- 
1.5.4.3