[Top][All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: computing filenames in skeletons
|
From: |
Joel E. Denny |
|
Subject: |
Re: computing filenames in skeletons |
|
Date: |
Tue, 12 Dec 2006 01:47:43 -0500 (EST) |
On Sat, 9 Dec 2006, Paul Eggert wrote:
> "Joel E. Denny" <address@hidden> writes:
>
> > + Make it easier for skeletons to compute output file names with the same
> > + file name prefix as Bison-computed output file names.
>
> Looks good to me. I assume you've tested
> it with weird file names?
As it turns out, not well enough. There are several existing bugs in this
area, and my proposed patch exercised them more. I have a new patch that
could probably use another pair of eyes.
Here's an interesting fact of which I was previously not aware: escapes do
not work within file names in #include's. As far as I can tell, it is
thus impossible to #include a file whose name contains both `>' and `"'.
Here's a reference:
http://www.ugcs.caltech.edu/manuals/devtool/cpp-3.4.5/cpp_7.html
Anyway, Bison previously used:
quotearg_style (c_quoting_style, FILE_NAME)
to quote the header file name for #include's in lalr1.cc and glr.c, and so
they were broken for header files with certain strange names. My proposed
patch ignorantly continued to do this.
Also, Bison previously did not escape M4-special characters in file names
before inserting them into the muscles. As a result, file names
containing `[' caused M4 errors. My proposed patch made `]' a problem as
well by trying to actually use those muscles in the skeletons.
I committed the following patch, which fixes the above and implements what
I was after originally after.
Since b4_pkgdatadir is used in m4_include, I haven't been able to come up
with a way to handle M4-special characters appearing inside the user's
installation path. Sucks to be such a user, I guess.
In case anyone is interested, `testsuite -l' botches test case titles (set
by AT_SETUP) containing `;'. One of my new tests cases has this problem,
but I don't believe it's vital for Bison that this be fixed. This should
probably be reported to bug-autoconf. Maybe later.
Index: ChangeLog
===================================================================
RCS file: /sources/bison/bison/ChangeLog,v
retrieving revision 1.1617
diff -p -u -r1.1617 ChangeLog
--- ChangeLog 10 Dec 2006 00:16:15 -0000 1.1617
+++ ChangeLog 12 Dec 2006 06:30:58 -0000
@@ -1,3 +1,45 @@
+2006-12-12 Joel E. Denny <address@hidden>
+
+ * tests/testsuite.at (AT_CHECK): When checking if $1 starts with
+ `bison ', use m4_index instead of m4_substr since chopping up a string
+ containing M4-special characters causes problems here.
+
+ Fix a couple of bugs related to special characters in user-specified
+ file names, and make it easier for skeletons to compute output file
+ names with the same file name prefix as Bison-computed output file
+ names.
+ * data/glr.cc, data/push.c, data/yacc.c: In @output, use
+ b4_parser_file_name and b4_spec_defines_file instead of
+ @output_parser_name@ and @output_header_name@, which are now redundant.
+ * data/glr.c, data/lalr1.cc: Likewise. Also, in header #include's, use
+ b4_parser_file_name, b4_spec_defines_file, and the new
+ @basename(FILENAME@) instead of @output_parser_name@ and
+ @output_header_name@, which inappropriately escaped the file names as
+ C string literals.
+ * src/files.c (all_but_ext): Remove static qualifier.
+ (compute_output_file_names): Move `free (all_but_ext)' to...
+ (output_file_names_free): ... here since all_but_ext is needed later.
+ * src/files.h (all_but_ext): Extern.
+ * src/muscle_tab.h (MUSCLE_INSERT_STRING_RAW): New macro that does
+ exactly what MUSCLE_INSERT_STRING used to do.
+ (MUSCLE_INSERT_STRING): Use MUSCLE_OBSTACK_SGROW so that M4-special
+ characters are escaped properly.
+ * src/output.c (prepare): Define muscle file_name_all_but_ext as
+ all_but_ext.
+ For pkgdatadir muscle, maintain previous functionality by using
+ MUSCLE_INSERT_STRING_RAW instead of MUSCLE_INSERT_STRING. The problem
+ is that b4_pkgdatadir is used inside m4_include in the skeletons, so
+ digraphs would never be expanded. Hopefully no one has M4-special
+ characters in his Bison installation path.
+ * src/scan-skel.l: Don't parse @output_header_name@ and
+ @output_parser_name@ anymore since they're now redundant.
+ In @output, use decode_at_digraphs.
+ Parse a new @basename command that invokes last_component.
+ (decode_at_digraphs): New.
+ (BASE_QPUTS): Remove unused.
+ * tests/output.at (AT_CHECK_OUTPUT_FILE_NAME): New macro.
+ (Output file name): New tests.
+
2006-12-09 Joel E. Denny <address@hidden>
Warn about output files that are generated by the skeletons and that
Index: data/glr.c
===================================================================
RCS file: /sources/bison/bison/data/glr.c,v
retrieving revision 1.192
diff -p -u -r1.192 glr.c
--- data/glr.c 10 Dec 2006 00:16:15 -0000 1.192
+++ data/glr.c 12 Dec 2006 06:30:59 -0000
@@ -142,7 +142,7 @@ m4_define([b4_rhs_location],
# We do want M4 expansion after # for CPP macros.
m4_changecom()
m4_divert(0)dnl
address@hidden @output_parser_name@
address@hidden b4_parser_file_name
b4_copyright([Skeleton implementation for Bison GLR parsers in C],
[2002, 2003, 2004, 2005, 2006])
[
@@ -207,7 +207,7 @@ typedef struct YYLTYPE
]b4_user_provides])[]dnl
])
-b4_defines_if([#include @address@hidden,
+b4_defines_if([[#include "@basename(]b4_spec_defines_file[@)"]],
[b4_shared_declarations])[
/* Enabling traces. */
@@ -2629,7 +2629,7 @@ dnl glr.cc produces its own header.
dnl
m4_if(b4_skeleton, ["glr.c"],
[b4_defines_if(
address@hidden @output_header_name@
address@hidden b4_spec_defines_file
b4_copyright([Skeleton interface for Bison GLR parsers in C],
[2002, 2003, 2004, 2005, 2006])
Index: data/glr.cc
===================================================================
RCS file: /sources/bison/bison/data/glr.cc,v
retrieving revision 1.28
diff -p -u -r1.28 glr.cc
--- data/glr.cc 15 Oct 2006 12:37:06 -0000 1.28
+++ data/glr.cc 12 Dec 2006 06:30:59 -0000
@@ -216,7 +216,7 @@ m4_defn([b4_parse_param])))],
m4_include(b4_pkgdatadir/[glr.c])
m4_popdef([b4_parse_param])
address@hidden @output_header_name@
address@hidden b4_spec_defines_file
b4_copyright([Skeleton interface for Bison GLR parsers in C++],
[2002, 2003, 2004, 2005, 2006])[
Index: data/lalr1.cc
===================================================================
RCS file: /sources/bison/bison/data/lalr1.cc,v
retrieving revision 1.145
diff -p -u -r1.145 lalr1.cc
--- data/lalr1.cc 15 Oct 2006 12:37:06 -0000 1.145
+++ data/lalr1.cc 12 Dec 2006 06:30:59 -0000
@@ -33,7 +33,7 @@ m4_include(b4_pkgdatadir/[location.cc])
m4_changecom()
m4_divert(0)dnl
b4_defines_if(
address@hidden @output_header_name@
address@hidden b4_spec_defines_file
b4_copyright([Skeleton interface for Bison LALR(1) parsers in C++],
[2002, 2003, 2004, 2005, 2006])
dnl FIXME: This is wrong, we want computed header guards.
@@ -301,7 +301,7 @@ m4_ifdef([b4_provides],
[#endif /* ! defined PARSER_HEADER_H */]
])dnl
address@hidden @output_parser_name@
address@hidden b4_parser_file_name
b4_copyright([Skeleton implementation for Bison LALR(1) parsers in C++],
[2002, 2003, 2004, 2005, 2006])
m4_if(b4_prefix, [yy], [],
@@ -312,8 +312,8 @@ m4_if(b4_prefix, [yy], [],
/* First part of user declarations. */
]b4_user_pre_prologue
-b4_defines_if([
-#include @address@hidden)[
+b4_defines_if([[
+#include "@basename(]b4_spec_defines_file[@)"]])[
/* User implementation prologue. */
]b4_user_post_prologue[
Index: data/push.c
===================================================================
RCS file: /sources/bison/bison/data/push.c,v
retrieving revision 1.11
diff -p -u -r1.11 push.c
--- data/push.c 8 Dec 2006 22:36:33 -0000 1.11
+++ data/push.c 12 Dec 2006 06:30:59 -0000
@@ -134,7 +134,7 @@ m4_define([b4_rhs_location],
# We do want M4 expansion after # for CPP macros.
m4_changecom()
m4_divert(0)dnl
address@hidden @output_parser_name@
address@hidden b4_parser_file_name
b4_copyright([Skeleton implementation for Bison's Yacc-like parsers in C],dnl '
[1984, 1989, 1990, 2000, 2001, 2002, 2003, 2004, 2005, 2006])[
@@ -1638,7 +1638,7 @@ yyreturn:
b4_epilogue
b4_defines_if(
address@hidden @output_header_name@
address@hidden b4_spec_defines_file
b4_copyright([Skeleton interface for Bison's Yacc-like parsers in C],dnl '
[1984, 1989, 1990, 2000, 2001, 2002, 2003, 2004, 2005, 2006])
Index: data/yacc.c
===================================================================
RCS file: /sources/bison/bison/data/yacc.c,v
retrieving revision 1.155
diff -p -u -r1.155 yacc.c
--- data/yacc.c 22 Oct 2006 00:28:16 -0000 1.155
+++ data/yacc.c 12 Dec 2006 06:31:00 -0000
@@ -134,7 +134,7 @@ m4_define([b4_rhs_location],
# We do want M4 expansion after # for CPP macros.
m4_changecom()
m4_divert(0)dnl
address@hidden @output_parser_name@
address@hidden b4_parser_file_name
b4_copyright([Skeleton implementation for Bison's Yacc-like parsers in C],dnl '
[1984, 1989, 1990, 2000, 2001, 2002, 2003, 2004, 2005, 2006])[
@@ -1485,7 +1485,7 @@ yyreturn:
b4_epilogue
b4_defines_if(
address@hidden @output_header_name@
address@hidden b4_spec_defines_file
b4_copyright([Skeleton interface for Bison's Yacc-like parsers in C],dnl '
[1984, 1989, 1990, 2000, 2001, 2002, 2003, 2004, 2005, 2006])
Index: src/files.c
===================================================================
RCS file: /sources/bison/bison/src/files.c,v
retrieving revision 1.100
diff -p -u -r1.100 files.c
--- src/files.c 10 Dec 2006 00:16:15 -0000 1.100
+++ src/files.c 12 Dec 2006 06:31:00 -0000
@@ -76,7 +76,7 @@ uniqstr current_file = NULL;
empty string (meaning the current directory); otherwise it is
`dir/'. */
-static char *all_but_ext;
+char *all_but_ext;
static char *all_but_tab_ext;
char *dir_prefix;
@@ -328,7 +328,6 @@ compute_output_file_names (void)
output_file_name_check (spec_verbose_file);
}
- free (all_but_ext);
free (all_but_tab_ext);
free (src_extension);
free (header_extension);
@@ -350,6 +349,7 @@ output_file_name_check (char const *file
void
output_file_names_free (void)
{
+ free (all_but_ext);
free (spec_verbose_file);
free (spec_graph_file);
free (spec_defines_file);
Index: src/files.h
===================================================================
RCS file: /sources/bison/bison/src/files.h,v
retrieving revision 1.43
diff -p -u -r1.43 files.h
--- src/files.h 10 Dec 2006 00:16:15 -0000 1.43
+++ src/files.h 12 Dec 2006 06:31:00 -0000
@@ -62,6 +62,9 @@ extern uniqstr grammar_file;
/* The current file name. Might change with %include, or with #line. */
extern uniqstr current_file;
+/* The computed base for output file names. */
+extern char *all_but_ext;
+
void compute_output_file_names (void);
void output_file_names_free (void);
void output_file_name_check (char const *file_name);
Index: src/muscle_tab.h
===================================================================
RCS file: /sources/bison/bison/src/muscle_tab.h,v
retrieving revision 1.18
diff -p -u -r1.18 muscle_tab.h
--- src/muscle_tab.h 8 Nov 2006 20:28:57 -0000 1.18
+++ src/muscle_tab.h 12 Dec 2006 06:31:00 -0000
@@ -52,13 +52,20 @@ do {
\
muscle_insert (Key, obstack_finish (&muscle_obstack)); \
} while(0)
-#define MUSCLE_INSERT_STRING(Key, Value) \
+#define MUSCLE_INSERT_STRING_RAW(Key, Value) \
do { \
obstack_sgrow (&muscle_obstack, Value); \
obstack_1grow (&muscle_obstack, 0); \
muscle_insert (Key, obstack_finish (&muscle_obstack)); \
} while(0)
+#define MUSCLE_INSERT_STRING(Key, Value) \
+do { \
+ MUSCLE_OBSTACK_SGROW (&muscle_obstack, Value); \
+ obstack_1grow (&muscle_obstack, 0); \
+ muscle_insert (Key, obstack_finish (&muscle_obstack)); \
+} while(0)
+
#define MUSCLE_OBSTACK_SGROW(Obstack, Value) \
do { \
char const *p; \
Index: src/output.c
===================================================================
RCS file: /sources/bison/bison/src/output.c,v
retrieving revision 1.259
diff -p -u -r1.259 output.c
--- src/output.c 12 Nov 2006 07:39:37 -0000 1.259
+++ src/output.c 12 Dec 2006 06:31:00 -0000
@@ -603,6 +603,8 @@ prepare (void)
if (spec_name_prefix)
MUSCLE_INSERT_STRING ("prefix", spec_name_prefix);
+ MUSCLE_INSERT_STRING ("file_name_all_but_ext", all_but_ext);
+
#define DEFINE(Name) MUSCLE_INSERT_STRING (#Name, Name ? Name : "")
DEFINE (dir_prefix);
DEFINE (parser_file_name);
@@ -632,7 +634,11 @@ prepare (void)
/* About the skeletons. */
{
char const *pkgdatadir = getenv ("BISON_PKGDATADIR");
- MUSCLE_INSERT_STRING ("pkgdatadir", pkgdatadir ? pkgdatadir : PKGDATADIR);
+ /* b4_pkgdatadir is used inside m4_include in the skeletons, so digraphs
+ would never be expanded. Hopefully no one has M4-special characters in
+ his Bison installation path. */
+ MUSCLE_INSERT_STRING_RAW ("pkgdatadir",
+ pkgdatadir ? pkgdatadir : PKGDATADIR);
MUSCLE_INSERT_C_STRING ("skeleton", skeleton);
}
}
Index: src/scan-skel.l
===================================================================
RCS file: /sources/bison/bison/src/scan-skel.l,v
retrieving revision 1.43
diff -p -u -r1.43 scan-skel.l
--- src/scan-skel.l 10 Dec 2006 00:16:15 -0000 1.43
+++ src/scan-skel.l 12 Dec 2006 06:31:00 -0000
@@ -47,9 +47,7 @@ int skel_lex (void);
#define QPUTS(String) \
fputs (quotearg_style (c_quoting_style, String), yyout)
-#define BASE_QPUTS(File) \
- QPUTS (last_component (File))
-
+static char *decode_at_digraphs (char *in);
%}
%%
@@ -59,29 +57,15 @@ int skel_lex (void);
%}
"@output ".*\n {
- char const *file_name = yytext + sizeof "@output " - 1;
+ char *file_name = yytext + sizeof "@output " - 1;
yytext[yyleng - 1] = '\0';
-
- /* Decode special file names. They include the directory part,
- contrary to their "free" occurrences, used for issuing #includes,
- which must not include the directory part. */
-
- if (*file_name == '@')
- {
- if (strcmp (file_name, "@output_header_name@") == 0)
- file_name = spec_defines_file;
- else if (strcmp (file_name, "@output_parser_name@") == 0)
- file_name = parser_file_name;
- else
- fatal ("invalid token in skeleton: %s", yytext);
- }
if (outname)
{
free (outname);
xfclose (yyout);
}
- outname = xstrdup (file_name);
- output_file_name_check (file_name);
+ outname = decode_at_digraphs (file_name);
+ output_file_name_check (outname);
yyout = xfopen (outname, "w");
lineno = 1;
}
@@ -93,8 +77,14 @@ int skel_lex (void);
"@oline@" fprintf (yyout, "%d", lineno + 1);
"@ofile@" QPUTS (outname);
"@dir_prefix@" QPUTS (dir_prefix);
-"@output_parser_name@" BASE_QPUTS (parser_file_name);
-"@output_header_name@" BASE_QPUTS (spec_defines_file);
+
+"@basename(".*"@)" {
+ char *file_name = yytext + sizeof "@basename " - 1;
+ yytext[yyleng - 2] = '\0';
+ file_name = decode_at_digraphs (file_name);
+ fputs (last_component (file_name), yyout);
+ free (file_name);
+}
/* This pattern must not match more than the previous @ patterns. */
@address@hidden fatal ("invalid @ in skeleton: %s", yytext);
@@ -124,3 +114,31 @@ scan_skel (FILE *in)
/* Reclaim Flex's buffers. */
yylex_destroy ();
}
+
+static char *
+decode_at_digraphs (char *in)
+{
+ char *out_start = xnmalloc (strlen (in)+1, sizeof *out_start);
+ char *out;
+ for (out = out_start; *in != '\0'; ++in, ++out)
+ {
+ if (*in == '@')
+ {
+ switch (*++in)
+ {
+ case '@': *out = '@'; break;
+ case '{': *out = '['; break;
+ case '}': *out = ']'; break;
+ default:
+ if (*in != '\0')
+ in[1] = '\0';
+ fatal ("invalid @ in skeleton: %s", in-1);
+ break;
+ }
+ }
+ else
+ *out = *in;
+ }
+ *out = '\0';
+ return out_start;
+}
Index: tests/output.at
===================================================================
RCS file: /sources/bison/bison/tests/output.at,v
retrieving revision 1.13
diff -p -u -r1.13 output.at
--- tests/output.at 10 Dec 2006 00:16:15 -0000 1.13
+++ tests/output.at 12 Dec 2006 06:31:00 -0000
@@ -23,7 +23,7 @@ AT_BANNER([[Output file names.]])
# [ADDITIONAL-TESTS])
# -----------------------------------------------------------------------------
m4_define([AT_CHECK_OUTPUT],
-[AT_SETUP([Output files: $2 $3 $5])
+[AT_SETUP([[Output files: $2 $3 $5]])
case "$1" in
*/*) mkdir `echo "$1" | sed 's,/.*,,'`;;
esac
@@ -149,3 +149,52 @@ AT_CHECK_CONFLICTING_OUTPUT([foo.y],
[%skeleton "lalr1.cc" %defines], [--graph="location.hh"],
[foo.y: warning: conflicting outputs to file `location.hh'
])
+
+
+# AT_CHECK_OUTPUT_FILE_NAME(FILE-NAME-PREFIX, [ADDITIONAL-TESTS])
+# -----------------------------------------------------------------------------
+m4_define([AT_CHECK_OUTPUT_FILE_NAME],
+[AT_SETUP([Output file name: $1])
+
+AT_DATA([glr.y],
+[[%glr-parser
+%code {
+ int yylex (void);
+ void yyerror (const char *);
+}
+%%
+start: {};
+]])
+AT_CHECK([bison -o "AS_ESCAPE([$1.c])" --defines="AS_ESCAPE([$1.h])" glr.y])
+AT_CHECK([ls "AS_ESCAPE([$1.c])" "AS_ESCAPE([$1.h])"], [], [ignore])
+AT_COMPILE([glr.o], [-c "AS_ESCAPE([$1.c])"])
+$2
+
+AT_DATA([cxx.y],
+[[%skeleton "lalr1.cc"
+%code { int yylex (yy::parser::semantic_type*); }
+%%
+start: {};
+]])
+AT_CHECK([bison -o "AS_ESCAPE([$1.c])" --defines="AS_ESCAPE([$1.h])" cxx.y])
+AT_CHECK([ls "AS_ESCAPE([$1.c])" "AS_ESCAPE([$1.h])"], [], [ignore])
+AT_COMPILE_CXX([cxx.o], [-c "AS_ESCAPE([$1.c])"])
+$2
+
+AT_CLEANUP
+])
+
+# Notice that the header file name here cannot contain
+# `"' since FILENAME in `#include "FILENAME"' cannot.
+AT_CHECK_OUTPUT_FILE_NAME(address@hidden&*()-=_+{}[]|\:;<>, .']])
+AT_CHECK_OUTPUT_FILE_NAME([[(]])
+AT_CHECK_OUTPUT_FILE_NAME([[)]])
+AT_CHECK_OUTPUT_FILE_NAME([[#]])
+AT_CHECK_OUTPUT_FILE_NAME([[@@]])
+AT_CHECK_OUTPUT_FILE_NAME(address@hidden)
+AT_CHECK_OUTPUT_FILE_NAME(address@hidden)
+
+# Bison isn't M4-quoting file names before inserting them into muscles, so
+# these tests currently fail.
+AT_CHECK_OUTPUT_FILE_NAME([[@<:@]])
+AT_CHECK_OUTPUT_FILE_NAME([[@:>@]])
Index: tests/testsuite.at
===================================================================
RCS file: /sources/bison/bison/tests/testsuite.at,v
retrieving revision 1.31
diff -p -u -r1.31 testsuite.at
--- tests/testsuite.at 11 Nov 2006 04:12:48 -0000 1.31
+++ tests/testsuite.at 12 Dec 2006 06:31:00 -0000
@@ -24,7 +24,7 @@
m4_pushdef([ORIGINAL_AT_CHECK], m4_defn([AT_CHECK]))
m4_pushdef([AT_CHECK],
[ORIGINAL_AT_CHECK(
- m4_if(m4_quote(m4_substr(m4_quote($1), 0, 5)), [bison],
+ m4_if(m4_index(m4_quote($1), [bison ]), 0,
m4_if(m4_quote($2), [0], [], m4_quote($2), [], [],
[[VALGRIND_OPTS="$VALGRIND_OPTS --leak-check=summary
--show-reachable=no"; export VALGRIND_OPTS; ]]))$@)])