[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[PATCH 02/10] api.token.raw: implement
From: |
Akim Demaille |
Subject: |
[PATCH 02/10] api.token.raw: implement |
Date: |
Sun, 1 Sep 2019 18:41:15 +0200 |
Bison used to feature %raw, documented as follows:
@item %raw
The output file @file{@var{name}.h} normally defines the tokens with
Yacc-compatible token numbers. If this option is specified, the
internal Bison numbers are used instead. (Yacc-compatible numbers start
at 257 except for single character tokens; Bison assigns token numbers
sequentially for all tokens starting at 3.)
Unfortunately, as far as I can tell, it never worked: token numbers
are indeed changed in the generated tables (from external token number
to internal), yet the code was still applying the mapping from
external token numbers to internal token numbers.
This commit reintroduces the feature as it was expected to be.
* data/skeletons/bison.m4 (b4_token_format): When api.token.raw is
enabled, use the internal token number.
* data/skeletons/yacc.c (yytranslate): Don't emit if api.token.raw is
enabled.
(YYTRANSLATE): Adjust.
---
data/skeletons/bison.m4 | 3 ++-
data/skeletons/yacc.c | 6 ++++--
2 files changed, 6 insertions(+), 3 deletions(-)
diff --git a/data/skeletons/bison.m4 b/data/skeletons/bison.m4
index 2c01ac0f..0fe0dc80 100644
--- a/data/skeletons/bison.m4
+++ b/data/skeletons/bison.m4
@@ -535,7 +535,7 @@ m4_define([b4_token_format],
[b4_token_visible_if([$2],
[m4_quote(m4_format([$1],
[b4_symbol([$2], [id])],
- [b4_symbol([$2], [user_number])]))])])
+ [b4_symbol([$2], b4_api_token_raw_if([[number]],
[[user_number]]))]))])])
## ------- ##
@@ -1002,6 +1002,7 @@ m4_define([b4_percent_code_ifdef],
# b4_parse_trace_if([IF-DEBUG-TRACES-ARE-ENABLED], [IF-NOT])
# b4_token_ctor_if([IF-YYLEX-RETURNS-A-TOKEN], [IF-NOT])
# ----------------------------------------------------------
+b4_percent_define_if_define([api.token.raw])
b4_percent_define_if_define([token_ctor], [api.token.constructor])
b4_percent_define_if_define([locations]) # Whether locations are tracked.
b4_percent_define_if_define([parse.assert])
diff --git a/data/skeletons/yacc.c b/data/skeletons/yacc.c
index 1de17886..ea97ad46 100644
--- a/data/skeletons/yacc.c
+++ b/data/skeletons/yacc.c
@@ -608,7 +608,9 @@ union yyalloc
/* YYTRANSLATE(TOKEN-NUM) -- Symbol number corresponding to TOKEN-NUM
as returned by yylex, with out-of-bounds checking. */
-#define YYTRANSLATE(YYX) \
+]b4_api_token_raw_if(dnl
+[[#define YYTRANSLATE(YYX) (YYX)]],
+[[#define YYTRANSLATE(YYX) \
((unsigned) (YYX) <= YYMAXUTOK ? yytranslate[YYX] : YYUNDEFTOK)
/* YYTRANSLATE[TOKEN-NUM] -- Symbol number corresponding to TOKEN-NUM
@@ -616,7 +618,7 @@ union yyalloc
static const ]b4_int_type_for([b4_translate])[ yytranslate[] =
{
]b4_translate[
-};
+};]])[
#if ]b4_api_PREFIX[DEBUG
]b4_integral_parser_table_define([rline], [b4_rline],
--
2.23.0
- RFC: Introduce api.token.raw, Akim Demaille, 2019/09/01
- [PATCH 01/10] style: tidy yacc.c, Akim Demaille, 2019/09/01
- [PATCH 02/10] api.token.raw: implement,
Akim Demaille <=
- [PATCH 03/10] api.token.raw: check it, Akim Demaille, 2019/09/01
- [PATCH 04/10] api.token.raw: apply to the other skeletons, Akim Demaille, 2019/09/01
- [PATCH 05/10] api.token.raw: cannot be used with character literals, Akim Demaille, 2019/09/01
- [PATCH 06/10] api.token.raw: document it, Akim Demaille, 2019/09/01
- [PATCH 07/10] parser: use api.token.raw, Akim Demaille, 2019/09/01
- [PATCH 09/10] d: handle eof in yytranslate, Akim Demaille, 2019/09/01
- [PATCH 10/10] java: handle eof in yytranslate, Akim Demaille, 2019/09/01
- [PATCH 08/10] regen, Akim Demaille, 2019/09/01
- Re: RFC: Introduce api.token.raw, Akim Demaille, 2019/09/14