gettext-0.12.1 msgfmt (built with bison 1.875) cannot read .po files

[Alexander Dupuy]

When building GNU gettext using bison 1.875, there is an inconsistency in the
token values between gettext-tools/src/po-gram-gen2.h and the po-gram-gen.h
that bison creates from po-gram-gen.y.

The po-gram-gen2.h defines the tokens as:

# define        COMMENT 257
# define        DOMAIN  258
# define        JUNK    259
# define        MSGID   260
# define        MSGID_PLURAL    261
# define        MSGSTR  262
# define        NAME    263
# define        NUMBER  264
# define        STRING  265

while the bison-generated file po-gram-gen.h defines them as:

#ifndef YYTOKENTYPE
# define YYTOKENTYPE
   /* Put the tokens into the symbol table, so that GDB and other debuggers
      know about them.  */
   enum yytokentype {
     COMMENT = 258,
     DOMAIN = 259,
     JUNK = 260,
     MSGID = 261,
     MSGID_PLURAL = 262,
     MSGSTR = 263,
     NAME = 264,
     NUMBER = 265,
     STRING = 266
   };
#endif
#define COMMENT 258
#define DOMAIN 259
#define JUNK 260
#define MSGID 261
#define MSGID_PLURAL 262
#define MSGSTR 263
#define NAME 264
#define NUMBER 265
#define STRING 266

This is a change in behavior between bison.1.875 and earlier 1.35, which
started the tokens at 257, but I think it is hard to argue that this is a "bug"
in bison.

In any case, if po-gram-gen2.h will be explicitly defining token values, the
bison grammar po-gram-gen.y should do so as well (with consistent values), and
this is what I have done as a workaround:

--- po-gram-gen.y.~1.1.1.1~     Mon Sep 15 10:43:39 2003
+++ po-gram-gen.y       Fri Sep 12 21:09:30 2003
@@ -109,16 +109,16 @@

 %}

-%token COMMENT
-%token DOMAIN
-%token JUNK
-%token MSGID
-%token MSGID_PLURAL
-%token MSGSTR
-%token NAME
+%token COMMENT 257
+%token DOMAIN  258
+%token JUNK    259
+%token MSGID   260
+%token MSGID_PLURAL    261
+%token MSGSTR  262
+%token NAME    263
 %token '[' ']'
-%token NUMBER
-%token STRING
+%token NUMBER  264
+%token STRING  265

 %union
 {

Better would be to eliminate use of po-gram-gen2.h and just use the token
values assigned by bison in po-gram-gen.h, but this is a more substantial
change, which didn't make sense for me.

@alex