Conflicts in typed token constructors with identical types

[Frank Heckenbach]

Hi,

the new typed token constructors produce conflicts if the same type
is spelled differently, e.g. in this parser where the difference is
just a space character:

%skeleton "lalr1.cc"
%define api.value.type variant
%define api.token.constructor
%code { yy::parser::symbol_type yylex (); }

%token <int> a;
%token <int > b;

%%
e: a b;
%%

yy::parser::symbol_type yylex ()
{
  static int n;
  switch (n++)
  {
    case 0: return { yy::parser::token::a, 1 };
    case 1: return { yy::parser::token::b, 2 };
    default: return 0;
  }
}

void yy::parser::error (const std::string &)
{
}

int main ()
{
  return yy::parser () ();
}

Bison produces conflicting declarations for "int" and "int ":

tt.cc:443:7: error: 'yy::parser::basic_symbol<Base>::basic_symbol(typename 
Base::kind_type, int&&)' cannot be overloaded
       basic_symbol (typename Base::kind_type t, int && v)
       ^~~~~~~~~~~~
tt.cc:432:7: error: with 'yy::parser::basic_symbol<Base>::basic_symbol(typename 
Base::kind_type, int&&)'
       basic_symbol (typename Base::kind_type t, int&& v)
       ^~~~~~~~~~~~
tt.cc:583:7: error: 'yy::parser::symbol_type::symbol_type(int, int)' cannot be 
overloaded
       symbol_type (int tok, int  v)
       ^~~~~~~~~~~
tt.cc:570:7: error: with 'yy::parser::symbol_type::symbol_type(int, int)'
       symbol_type (int tok, int v)
       ^~~~~~~~~~~

This particular case could be solved be normalizing spellings
somewhat, such as removing leading/trailing spaces and converting
multiple inner spaces/tabs to a single space.

But of course, in the presence of typedef/using, same types can
appear different. In fact, it may depend on external headers whether
two types are the same, and this may not even be known at
Bison-time, but vary at compile time.

E.g. consider "int" vs. "int32_t" which are the same on some systems
and different on others. If a Bison-generated parser is shipped in a
source distribution (as Bison itself does) and used on different
systems, this difference can become relevant.

For an easily testable example, see this slightly modified version
of the example above (where the conditional setting of foo could
easily be imagined to be in an external header). As of now, it works
with "#if 0" (int vs. short) and fails with "#if 1" (int vs. int):

%skeleton "lalr1.cc"
%define api.value.type variant
%define api.token.constructor
%code requires
{
#if 1
using foo = int;
#else
using foo = short;
#endif
}
%code { yy::parser::symbol_type yylex (); }

%token <int> a;
%token <foo> b;

%%
e: a b;
%%

yy::parser::symbol_type yylex ()
{
  static int n;
  switch (n++)
  {
    case 0: return { yy::parser::token::a, 1 };
    case 1: return { yy::parser::token::b, foo (2) };
    default: return 0;
  }
}

void yy::parser::error (const std::string &)
{
}

int main ()
{
  return yy::parser () ();
}

As said above, it can't be solved at Bison-time, only at
compile-time. Fortunately, SFINAE (enable_if) seems to provide a
solution to avoid generating the overload if it would actually be a
duplicate.

Furthermore, since the first overload will then be used for both
cases, its assertion needs to accept both of them.

So with these manual changes as indicated to the generated parser,
the above example works in both cases:

=>    template <typename T = foo, typename = std::enable_if_t <!std::is_same 
<T, int>::value>>
      symbol_type (int tok, foo v)
        : super_type(token_type (tok), std::move (v))
      {
        YYASSERT (tok == token::b);
      }

      symbol_type (int tok, int v)
        : super_type(token_type (tok), std::move (v))
      {
=>      YYASSERT (tok == token::a || (std::is_same <int, foo>::value && tok == 
token::b));
      }

=>    template <typename T = foo, typename = std::enable_if_t <!std::is_same 
<T, int>::value>>
      symbol_type (int tok, foo v)
        : super_type(token_type (tok), std::move (v))
      {
        YYASSERT (tok == token::b);
      }

It gets uglier than it looks in this simple case as the number of
is_same checks grows quadratically with the number of types in the
variant. Since generally each type needs to be compared with each
other type for sameness, I think that's unavoidable.

Of course, we could also take the easy way out and declare "don't do
that", since the practical use of types that may or may not be the
same seems questionable. Though it would be awkward when the
original developer uses a system where the types are different and
never knows about the potential issue until someone tries to build
the code on another system where they are the same.

I should mention that this is not an urgent problem to me. I just
stumbled upon a situation as in the original example by accidentally
putting a space too many (and after removing it, my code works
again), but kept thinking about the ramifications.

Regards,
Frank