[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[PATCH v4] Add command line option to map file prefixes
|
From: |
Joshua Watt |
|
Subject: |
[PATCH v4] Add command line option to map file prefixes |
|
Date: |
Sat, 23 May 2020 20:26:17 -0500 |
Teaches bison about a new command line option, --file-prefix-map OLD=NEW
(based on the -ffile-prefix-map option from GCC) which causes it to
replace and file path of OLD in the text of the output file with NEW,
mainly for header guards and comments. The primary use of this is to
make builds reproducible with different input paths, and in particular
the debugging information produced when the source code is compiled. For
example, a distro may know that the bison source code will be located at
"/usr/src/bison" and thus can generate bison files that are reproducible
with the following command:
bison --output=/build/bison/parse.c -d -L C
--file-prefix-map=/build/bison/=/usr/src/bison/ parse.y
Importantly, this will change the header guards and #line directives
from:
#ifndef YY_BUILD_BISON_PARSE_H
#line 100 "/build/bison/parse.h"
to
#ifndef YY_USR_SRC_BISON_PARSE_H
#line 100 "/usr/src/bison/parse.h"
which is reproducible.
Signed-off-by: Joshua Watt <address@hidden>
---
NEWS | 10 ++++
data/skeletons/glr.c | 4 +-
data/skeletons/glr.cc | 4 +-
data/skeletons/lalr1.cc | 6 +--
data/skeletons/location.cc | 2 +-
data/skeletons/yacc.c | 4 +-
doc/bison.texi | 4 ++
src/files.c | 84 ++++++++++++++++++++++++++++++
src/files.h | 9 ++++
src/getargs.c | 54 +++++++++++++-------
src/output.c | 2 +
src/scan-skel.l | 11 +++-
tests/input.at | 28 ++++++++++
tests/output.at | 101 +++++++++++++++++++++++++++++++++++++
14 files changed, 295 insertions(+), 28 deletions(-)
diff --git a/NEWS b/NEWS
index a807b545..8fc26618 100644
--- a/NEWS
+++ b/NEWS
@@ -7,6 +7,16 @@ GNU Bison NEWS
When installed to be relocatable (via configure --enable-relocatable),
bison will now also look for a relocated m4.
+** New features
+
+*** File prefix mapping
+
+ Bison learned a new argument, '--file-prefix-map OLD=NEW'. Any file path in
+ the output (specifically #line directives and #ifdef header guards) that
+ being with the prefix OLD will have it replace with the prefix NEW, similar
+ to the -ffile-prefix-map in GCC. This option can be used to make bison output
+ reproducible.
+
* Noteworthy changes in release 3.6.2 (2020-05-17) [stable]
** Bug fixes
diff --git a/data/skeletons/glr.c b/data/skeletons/glr.c
index 9a03fb2c..1f877988 100644
--- a/data/skeletons/glr.c
+++ b/data/skeletons/glr.c
@@ -189,9 +189,9 @@ b4_glr_cc_if([],
[b4_output_begin([b4_spec_header_file])
b4_copyright([Skeleton interface for Bison GLR parsers in C],
[2002-2015, 2018-2020])[
-]b4_cpp_guard_open([b4_spec_header_file])[
+]b4_cpp_guard_open([b4_spec_mapped_header_file])[
]b4_shared_declarations[
-]b4_cpp_guard_close([b4_spec_header_file])[
+]b4_cpp_guard_close([b4_spec_mapped_header_file])[
]b4_output_end[
]])])
diff --git a/data/skeletons/glr.cc b/data/skeletons/glr.cc
index a294fb20..374e5e83 100644
--- a/data/skeletons/glr.cc
+++ b/data/skeletons/glr.cc
@@ -370,9 +370,9 @@ b4_copyright([Skeleton interface for Bison GLR parsers in
C++],
// C++ GLR parser skeleton written by Akim Demaille.
]b4_disclaimer[
-]b4_cpp_guard_open([b4_spec_header_file])[
+]b4_cpp_guard_open([b4_spec_mapped_header_file])[
]b4_shared_declarations[
-]b4_cpp_guard_close([b4_spec_header_file])[
+]b4_cpp_guard_close([b4_spec_mapped_header_file])[
]b4_output_end])
# Let glr.c (and b4_shared_declarations) believe that the user
diff --git a/data/skeletons/lalr1.cc b/data/skeletons/lalr1.cc
index 7b6cc715..45d0626b 100644
--- a/data/skeletons/lalr1.cc
+++ b/data/skeletons/lalr1.cc
@@ -513,16 +513,16 @@ b4_defines_if(
b4_copyright([Skeleton interface for Bison LALR(1) parsers in C++])
[
/**
- ** \file ]b4_spec_header_file[
+ ** \file ]b4_spec_mapped_header_file[
** Define the ]b4_namespace_ref[::parser class.
*/
// C++ LALR(1) parser skeleton written by Akim Demaille.
]b4_disclaimer[
-]b4_cpp_guard_open([b4_spec_header_file])[
+]b4_cpp_guard_open([b4_spec_mapped_header_file])[
]b4_shared_declarations(hh)[
-]b4_cpp_guard_close([b4_spec_header_file])[
+]b4_cpp_guard_close([b4_spec_mapped_header_file])[
]b4_output_end[
]])
diff --git a/data/skeletons/location.cc b/data/skeletons/location.cc
index 34db4188..dff984e7 100644
--- a/data/skeletons/location.cc
+++ b/data/skeletons/location.cc
@@ -48,7 +48,7 @@ m4_ifdef([b4_location_file],
["b4_location_file"])])
m4_define([b4_location_path],
b4_percent_define_get([[api.location.include]],
- ["b4_dir_prefix[]b4_location_file"]))
+ ["b4_mapped_dir_prefix[]b4_location_file"]))
m4_define([b4_location_path],
m4_substr(m4_defn([b4_location_path]), 1,
m4_eval(m4_len(m4_defn([b4_location_path])) - 2)))
])
diff --git a/data/skeletons/yacc.c b/data/skeletons/yacc.c
index 8efb0a50..b82029c0 100644
--- a/data/skeletons/yacc.c
+++ b/data/skeletons/yacc.c
@@ -313,14 +313,14 @@ m4_define([b4_declare_yyparse],
# Declaration that might either go into the header (if --defines)
# or open coded in the parser body.
m4_define([b4_shared_declarations],
-[b4_cpp_guard_open([b4_spec_header_file])[
+[b4_cpp_guard_open([b4_spec_mapped_header_file])[
]b4_declare_yydebug[
]b4_percent_code_get([[requires]])[
]b4_token_enums_defines[
]b4_declare_yylstype[
]b4_declare_yyparse[
]b4_percent_code_get([[provides]])[
-]b4_cpp_guard_close([b4_spec_header_file])[]dnl
+]b4_cpp_guard_close([b4_spec_mapped_header_file])[]dnl
])
diff --git a/doc/bison.texi b/doc/bison.texi
index 2cbe1a69..2e002294 100644
--- a/doc/bison.texi
+++ b/doc/bison.texi
@@ -11280,6 +11280,10 @@ Output an XML report of the parser's automaton
computed by Bison.
@code{@var{file}} is optional.
If omitted and the grammar file is @file{foo.y}, the output file will be
@file{foo.xml}.
+
+@item -M @var{old}=@var{new}
+@itemx --file-prefix-map=@var{old}=@var{new}
+Replace prefix @var{old} with @var{new} when writing file paths in output files
@end table
@node Option Cross Key
diff --git a/src/files.c b/src/files.c
index 5b743ab5..6dca1cb9 100644
--- a/src/files.c
+++ b/src/files.c
@@ -37,6 +37,8 @@
#include "complain.h"
#include "files.h"
#include "getargs.h"
+#include "gl_array_list.h"
+#include "gl_xlist.h"
#include "gram.h"
/* Initializing some values below (such SPEC_NAME_PREFIX to 'yy') is
@@ -55,6 +57,7 @@ char *spec_verbose_file = NULL; /* for --verbose. */
char *spec_graph_file = NULL; /* for -g. */
char *spec_xml_file = NULL; /* for -x. */
char *spec_header_file = NULL; /* for --defines. */
+char *spec_mapped_header_file = NULL;
char *parser_file_name;
/* All computed output file names. */
@@ -91,11 +94,20 @@ uniqstr grammar_file = NULL;
char *all_but_ext;
static char *all_but_tab_ext;
char *dir_prefix;
+char *mapped_dir_prefix;
/* C source file extension (the parser source). */
static char *src_extension = NULL;
/* Header file extension (if option '`-d'' is specified). */
static char *header_extension = NULL;
+
+struct prefix_map
+{
+ char *oldprefix;
+ char *newprefix;
+};
+
+static gl_list_t prefix_maps = NULL;
/*-----------------------------------------------------------------.
| Return a newly allocated string composed of the concatenation of |
@@ -159,6 +171,70 @@ xfdopen (int fd, char const *mode)
return res;
}
+/* Given an input file path, returns a dynamically allocated string that
+ contains the path with the file prefix mapping rules applied, or NULL if
+ the input was NULL. */
+char *
+map_file_name (char const *filename)
+{
+ if (!filename)
+ return NULL;
+
+ struct prefix_map const *p = NULL;
+ if (prefix_maps)
+ {
+ void const *ptr;
+ gl_list_iterator_t iter = gl_list_iterator (prefix_maps);
+ while (gl_list_iterator_next (&iter, &ptr, NULL))
+ {
+ p = ptr;
+ if (strncmp (p->oldprefix, filename, strlen (p->oldprefix)) == 0)
+ break;
+ p = NULL;
+ }
+ gl_list_iterator_free (&iter);
+ }
+
+ if (!p)
+ return xstrdup (filename);
+
+ size_t oldprefix_len = strlen (p->oldprefix);
+ size_t newprefix_len = strlen (p->newprefix);
+ char *s = xmalloc (newprefix_len + strlen (filename) - oldprefix_len + 1);
+
+ char *end = stpcpy (s, p->newprefix);
+ stpcpy (end, filename + oldprefix_len);
+
+ return s;
+}
+
+static void
+prefix_map_free (struct prefix_map *p)
+{
+ free (p->oldprefix);
+ free (p->newprefix);
+ free (p);
+}
+
+/* Adds a new file prefix mapping. If a file path starts with oldprefix, it
+ will be replaced with newprefix */
+void
+add_prefix_map(char const* oldprefix, char const* newprefix)
+{
+ if (!prefix_maps)
+ prefix_maps = gl_list_create_empty (GL_ARRAY_LIST,
+ /* equals */ NULL,
+ /* hashcode */ NULL,
+ (gl_listelement_dispose_fn)
prefix_map_free,
+ true);
+
+ struct prefix_map *p = xmalloc (sizeof (*p));
+ p->oldprefix = xstrdup (oldprefix);
+ p->newprefix = xstrdup (newprefix);
+
+ gl_list_add_last (prefix_maps, p);
+}
+
/*------------------------------------------------------------------.
| Compute ALL_BUT_EXT, ALL_BUT_TAB_EXT and output files extensions. |
`------------------------------------------------------------------*/
@@ -363,6 +439,9 @@ compute_output_file_names (void)
output_file_name_check (&spec_verbose_file, false);
}
+ spec_mapped_header_file = map_file_name (spec_header_file);
+ mapped_dir_prefix = map_file_name (dir_prefix);
+
free (all_but_tab_ext);
free (src_extension);
free (header_extension);
@@ -449,10 +528,15 @@ output_file_names_free (void)
free (spec_graph_file);
free (spec_xml_file);
free (spec_header_file);
+ free (spec_mapped_header_file);
free (parser_file_name);
free (dir_prefix);
+ free (mapped_dir_prefix);
for (int i = 0; i < generated_files_size; i++)
free (generated_files[i].name);
free (generated_files);
free (relocate_buffer);
+
+ if (prefix_maps)
+ gl_list_free (prefix_maps);
}
diff --git a/src/files.h b/src/files.h
index 64b6f8b5..17b434cf 100644
--- a/src/files.h
+++ b/src/files.h
@@ -50,9 +50,15 @@ extern char *spec_xml_file;
/* File name specified with --defines. */
extern char *spec_header_file;
+/* File name specified with --defines, adjusted for mapped prefixes. */
+extern char *spec_mapped_header_file;
+
/* Directory prefix of output file names. */
extern char *dir_prefix;
+/* Directory prefix of output file name, adjusted for mapped prefixes. */
+extern char *mapped_dir_prefix;
+
/* The file name as given on the command line.
Not named "input_file" because Flex uses this name for an argument,
and therefore GCC warns about a name clash. */
@@ -85,4 +91,7 @@ FILE *xfopen (const char *name, char const *mode);
void xfclose (FILE *ptr);
FILE *xfdopen (int fd, char const *mode);
+char *map_file_name (char const *filename);
+void add_prefix_map(char const* oldprefix, char const* newprefix);
+
#endif /* !FILES_H_ */
diff --git a/src/getargs.c b/src/getargs.c
index cee488c2..afbc2bb4 100644
--- a/src/getargs.c
+++ b/src/getargs.c
@@ -423,15 +423,17 @@ Tuning the Parser:\n\
* won't assume that -d also takes an argument. */
fputs (_("\
Output Files:\n\
- --defines[=FILE] also produce a header file\n\
- -d likewise but cannot specify FILE (for POSIX
Yacc)\n\
- -r, --report=THINGS also produce details on the automaton\n\
- --report-file=FILE write report to FILE\n\
- -v, --verbose same as '--report=state'\n\
- -b, --file-prefix=PREFIX specify a PREFIX for output files\n\
- -o, --output=FILE leave output to FILE\n\
- -g, --graph[=FILE] also output a graph of the automaton\n\
- -x, --xml[=FILE] also output an XML report of the automaton\n\
+ --defines[=FILE] also produce a header file\n\
+ -d likewise but cannot specify FILE (for POSIX
Yacc)\n\
+ -r, --report=THINGS also produce details on the automaton\n\
+ --report-file=FILE write report to FILE\n\
+ -v, --verbose same as '--report=state'\n\
+ -b, --file-prefix=PREFIX specify a PREFIX for output files\n\
+ -o, --output=FILE leave output to FILE\n\
+ -g, --graph[=FILE] also output a graph of the automaton\n\
+ -x, --xml[=FILE] also output an XML report of the automaton\n\
+ -M, --file-prefix-map=OLD=NEW replace prefix OLD with NEW when writing file
paths\n\
+ in output files\n\
"), stdout);
putc ('\n', stdout);
@@ -552,6 +554,7 @@ static char const short_options[] =
"h"
"k"
"l"
+ "M:"
"o:"
"p:"
"r:"
@@ -603,14 +606,15 @@ static struct option const long_options[] =
{ "yacc", no_argument, 0, 'y' },
/* Output Files. */
- { "defines", optional_argument, 0, 'd' },
- { "report", required_argument, 0, 'r' },
- { "report-file", required_argument, 0, REPORT_FILE_OPTION },
- { "verbose", no_argument, 0, 'v' },
- { "file-prefix", required_argument, 0, 'b' },
- { "output", required_argument, 0, 'o' },
- { "graph", optional_argument, 0, 'g' },
- { "xml", optional_argument, 0, 'x' },
+ { "defines", optional_argument, 0, 'd' },
+ { "report", required_argument, 0, 'r' },
+ { "report-file", required_argument, 0, REPORT_FILE_OPTION },
+ { "verbose", no_argument, 0, 'v' },
+ { "file-prefix", required_argument, 0, 'b' },
+ { "output", required_argument, 0, 'o' },
+ { "graph", optional_argument, 0, 'g' },
+ { "xml", optional_argument, 0, 'x' },
+ { "file-prefix-map", required_argument, 0, 'M' },
/* Hidden. */
{ "fixed-output-files", no_argument, 0, FIXED_OUTPUT_FILES_OPTION },
@@ -716,6 +720,22 @@ getargs (int argc, char *argv[])
language_argmatch (optarg, command_line_prio, loc);
break;
+ case 'M': // -MOLDPREFIX=NEWPREFIX
+ {
+ char *newprefix = strchr (optarg, '=');
+ if (newprefix)
+ {
+ *newprefix = '\0';
+ add_prefix_map (optarg, newprefix + 1);
+ }
+ else
+ {
+ complain (&loc, complaint, _("invalid argument for %s: %s"),
+ quote ("--file-prefix-map"), quotearg_n (1, optarg));
+ }
+ }
+ break;
+
case 'S':
skeleton_arg (optarg, command_line_prio, loc);
break;
diff --git a/src/output.c b/src/output.c
index 8e3bb38c..11f57f8d 100644
--- a/src/output.c
+++ b/src/output.c
@@ -805,8 +805,10 @@ prepare (void)
#define DEFINE(Name) MUSCLE_INSERT_STRING (#Name, Name ? Name : "")
DEFINE (dir_prefix);
+ DEFINE (mapped_dir_prefix);
DEFINE (parser_file_name);
DEFINE (spec_header_file);
+ DEFINE (spec_mapped_header_file);
DEFINE (spec_file_prefix);
DEFINE (spec_graph_file);
DEFINE (spec_name_prefix);
diff --git a/src/scan-skel.l b/src/scan-skel.l
index a856eac0..703c56f5 100644
--- a/src/scan-skel.l
+++ b/src/scan-skel.l
@@ -50,6 +50,7 @@ static void at_output (int argc, char *argv[], char **name,
int *lineno);
static void fail_for_at_directive_too_many_args (char const
*at_directive_name);
static void fail_for_at_directive_too_few_args (char const *at_directive_name);
static void fail_for_invalid_at (char const *at);
+static void output_mapped_file (char const *name);
%}
%x SC_AT_DIRECTIVE_ARGS
@@ -77,7 +78,7 @@ static void fail_for_invalid_at (char const *at);
@\n continue;
"@oline@" fprintf (yyout, "%d", out_lineno + 1);
-"@ofile@" fputs (quotearg_style (c_quoting_style, out_name), yyout);
+"@ofile@" output_mapped_file (out_name);
"@basename(" at_init (&argc, argv, &at_ptr, &at_basename);
"@complain(" at_init (&argc, argv, &at_ptr, &at_complain);
@@ -264,3 +265,11 @@ fail_for_invalid_at (char const *at)
{
complain (NULL, fatal, "invalid @ in skeleton: %s", at);
}
+
+static void
+output_mapped_file (char const *name)
+{
+ char *f = map_file_name (name);
+ fputs (quotearg_style (c_quoting_style, f), yyout);
+ free (f);
+}
diff --git a/tests/input.at b/tests/input.at
index e0cf5f01..f374c07f 100644
--- a/tests/input.at
+++ b/tests/input.at
@@ -3006,3 +3006,31 @@ AT_TEST([%define parse.error detailed
%token-table],[[input.y:1.1-28]])
m4_popdef([AT_TEST])
AT_CLEANUP
+
+## ------------------------------------------------------ ##
+## invalid file prefix mapping arguments
+## ------------------------------------------------------ ##
+
+AT_SETUP([[invalid file prefix mapping arguments]])
+
+# AT_TEST(DIRECTIVES, OPTIONS, ERROR-LOCATION)
+# --------------------------------------------
+m4_pushdef([AT_TEST],
+[AT_DATA([[input.y]],
+[[
+%%
+exp: %empty;
+]])
+AT_BISON_CHECK([[$1 input.y]], [[1]], [[]],
+[[$3: error: invalid argument for '--file-prefix-map': $2
+]])
+])
+
+AT_TEST([-M foo], [foo], [<command line>:4])
+AT_TEST([--file-prefix-map foo], [foo], [<command line>:4])
+AT_TEST([-M foo=bar -M baz], [baz], [<command line>:6])
+AT_TEST([-M foo= -M baz], [baz], [<command line>:6])
+
+m4_popdef([AT_TEST])
+
+AT_CLEANUP
diff --git a/tests/output.at b/tests/output.at
index e7a1d14e..4802f536 100644
--- a/tests/output.at
+++ b/tests/output.at
@@ -674,3 +674,104 @@ imm: '0';
]])
m4_popdef([AT_TEST])
+
+
+## ------------------------------- ##
+## C++ Output File Prefix Mapping ##
+## ------------------------------- ##
+
+AT_SETUP([C++ Output File Prefix Mapping])
+
+# AT_TEST([PREFIX], [DIRECTIVES])
+m4_pushdef([AT_TEST],
+[AT_BISON_OPTION_PUSHDEFS([%skeleton "lalr1.cc" %define api.namespace {$1} $2])
+AT_LOC_PUSHDEF([begin.line], [begin.column], [end.line], [end.column])
+AT_DATA_GRAMMAR([$1.yy],
+[[%skeleton "lalr1.cc"
+%define api.namespace {$1}
+$2
+%code {
+ ]AT_YYERROR_DECLARE[
+ ]AT_YYLEX_DECLARE[
+}
+%%
+exp: '0';
+%%
+]AT_YYERROR_DEFINE[
+]AT_YYLEX_DEFINE(["0"])[
+]])
+
+AT_BISON_CHECK([-fcaret -o out/$1.cc -M out/=bar/ $1.yy])
+AT_LANG_COMPILE([out/$1.o], [], [-Iout/include])
+
+AT_LOC_POPDEF
+AT_BISON_OPTION_POPDEFS
+])
+
+mkdir -p out/include/ast
+
+AT_TEST([x1],
+ [%defines
+ %locations
+ %define api.location.file "include/ast/loc.hh"
+ ])
+
+# Check the CPP guard and Doxyen comments.
+AT_CHECK([sed -ne 's/#line [0-9]\+ "/#line "/p;/INCLUDED/p;/\\file/{p;n;p;}'
out/include/ast/loc.hh], [],
+[[ ** \file bar/include/ast/loc.hh
+ ** Define the x1::location class.
+#ifndef YY_YY_BAR_INCLUDE_AST_LOC_HH_INCLUDED
+# define YY_YY_BAR_INCLUDE_AST_LOC_HH_INCLUDED
+#line "x1.yy"
+#line "bar/include/ast/loc.hh"
+#line "x1.yy"
+#line "bar/include/ast/loc.hh"
+#endif // !YY_YY_BAR_INCLUDE_AST_LOC_HH_INCLUDED
+]])
+
+AT_CHECK([sed -ne 's/^#line [0-9]\+ "/#line "/p;/INCLUDED/p;/\\file/{p;n;p;}'
out/x1.hh], [],
+[[ ** \file bar/x1.hh
+ ** Define the x1::parser class.
+#ifndef YY_YY_BAR_X1_HH_INCLUDED
+# define YY_YY_BAR_X1_HH_INCLUDED
+#line "x1.yy"
+#line "bar/x1.hh"
+#line "x1.yy"
+#line "bar/x1.hh"
+#endif // !YY_YY_BAR_X1_HH_INCLUDED
+]])
+
+AT_TEST([x2],
+ [%defines
+ %locations
+ %code requires {#include "include/ast/loc.hh"}
+ %define api.location.type {x1::location}])
+
+m4_popdef([AT_TEST])
+
+AT_DATA([main.cc],
+[AT_DATA_SOURCE_PROLOGUE
+[#include "x1.hh"
+#include "x2.hh"
+
+#define RUN(S) \
+ do { \
+ S::parser parser; \
+ int res = parser.parse(); \
+ if (res) \
+ std::cerr << #S": " << res << '\n'; \
+ } while (false)
+
+int
+main (void)
+{
+ RUN(x1);
+ RUN(x2);
+}
+]])# main.cc
+
+AT_COMPILE_CXX([parser], [[out/x[12].o main.cc]], [-Iout/])
+AT_PARSER_CHECK([parser], [0])
+
+AT_CLEANUP
+
--
2.25.2
- [PATCH v4] Add command line option to map file prefixes,
Joshua Watt <=