m4-patches
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

FYI: 26-gary-trace-semantics-sanity.patch


From: Gary V. Vaughan
Subject: FYI: 26-gary-trace-semantics-sanity.patch
Date: Thu, 11 Oct 2001 22:30:26 +0100
User-agent: Mutt/1.3.16i

Here it is at last... the saner trace semantics tha Akim asked for :-)
I couldn't resist the temptation to do another rewrite of the symbol table
handling, mainly a refactoring to reduce the depth of the C call stack and to
abstract the special status of an undefined symbol with the trace bit set.
Also the symbols defined by m4private.h are not exported outside of the m4
sources, so there is no need to enforce the `m4_' namespace.

Interestingly, both of the M4 implementations on solaris simply lose the trace
bit if you undefine the symbol it is attached to.  I decided to make GNU M4
bugwards compatible if the -G option is passed.

Enjoy!
        Gary.

Index: ChangeLog
from  Gary V. Vaughan  <address@hidden>
        The trace semantics now attach the trace bit to a symbol name.
        For as long as a traceon(`foo') is active, calls to foo will be
        traced regardless of intervening undefines or module unloads.
        Fixed the flag propogation issues differently to the fixes
        reverted with the last attempt at nailing down trace:

        * m4/m4private.h:  This file is not visible outside of the m4
        source tree, so I removed all the `M4_' and `m4_' prefixes to save
        on typing.  Updated all clients.
        (m4_token): New typedef contains the fields that need to be passed
        around with the low level tokeniser.
        (m4_token_data):  Removed.
        (m4_symbol): Now contains the traced flag again.
        * m4/symtab.c:  Rewritten again.  Now we don't remove symbols with
        the trace bit set.  This change is contained entirely within this
        module and doesn't leak out into the callers.  Updated all
        clients.
        (m4_symbol_builtin, m4_symbol_macro):  Adjusted to make use of the
        new fields in m4_token.  Updated all clients.
        * m4/builtin.c (m4_builtin_pushdef):  Needs a flags argument so
        that `groks_macro_args' and `blind_if_no_args' flags are retained
        when defn() results are passed around in m4 code.  Updated all
        callers.
        (m4_builtin_define): Ditto.
        (m4_macro_pushdef, m4_macro_define): Ditto.
        * m4/input.c (struct input_block): Add a flags field to facilitate
        the above.
        (m4_push_macro):  ...use it.
        (init_macro_token):  Retrieve it.
        (m4_next_token): Initialise it.
        * m4/macro.c (expand_argument):  Copy it.
        * m4/m4module.h: Adjust.
        * tests/builtins.at:  Adjust the expected output of the trace
        tests to reflect the change in semantics.
        * tests/modules.at (modules: trace):  Check that unloading a
        module which supplies a traced symbol definition doesn't lose te
        trace bit.

Index: m4/builtin.c
===================================================================
RCS file: /cvsroot/m4/m4/m4/builtin.c,v
retrieving revision 1.12
diff -u -p -u -r1.12 builtin.c
--- m4/builtin.c 2001/09/08 01:56:34 1.12
+++ m4/builtin.c 2001/10/11 21:01:00
@@ -72,36 +72,36 @@ m4_builtin_find_by_func (const m4_builti
 
 m4_symbol *
 m4_builtin_pushdef (const char *name, lt_dlhandle handle,
-                   const m4_builtin *bp)
+                   m4_builtin_func *func, int flags)
 {
   m4_symbol *symbol;
 
   assert (name);
   assert (handle);
-  assert (bp);
+  assert (func);
 
   symbol = m4_symbol_pushdef (name);
 
   if (symbol)
-    m4_symbol_builtin (symbol, handle, bp);
+    m4_symbol_builtin (symbol, handle, func, flags);
 
   return symbol;
 }
 
 m4_symbol *
 m4_builtin_define (const char *name, lt_dlhandle handle,
-                   const m4_builtin *bp)
+                  m4_builtin_func *func, int flags)
 {
   m4_symbol *symbol;
 
   assert (name);
   assert (handle);
-  assert (bp);
+  assert (func);
 
   symbol = m4_symbol_define (name);
 
   if (symbol)
-    m4_symbol_builtin (symbol, handle, bp);
+    m4_symbol_builtin (symbol, handle, func, flags);
 
   return symbol;
 }
@@ -116,6 +116,7 @@ m4_builtin_table_install (lt_dlhandle ha
 
   for (bp = table; bp->name != NULL; bp++)
     {
+      int flags = 0;
       char *key;
 
       if (prefix_all_builtins)
@@ -129,15 +130,19 @@ m4_builtin_table_install (lt_dlhandle ha
       else
        key = (char *) bp->name;
 
-      m4_builtin_pushdef (key, handle, bp);
+      if (bp->groks_macro_args) BIT_SET (flags, TOKEN_MACRO_ARGS_BIT);
+      if (bp->blind_if_no_args) BIT_SET (flags, TOKEN_BLIND_ARGS_BIT);
 
+      m4_builtin_pushdef (key, handle, bp->func, flags);
+
       if (prefix_all_builtins)
        xfree (key);
     }
 }
 
 m4_symbol *
-m4_macro_pushdef (const char *name, lt_dlhandle handle, const char *text)
+m4_macro_pushdef (const char *name, lt_dlhandle handle, const char *text,
+                 int flags)
 {
   m4_symbol *symbol;
 
@@ -147,13 +152,14 @@ m4_macro_pushdef (const char *name, lt_d
   symbol = m4_symbol_pushdef (name);
 
   if (symbol)
-    m4_symbol_macro (symbol, handle, text);
+    m4_symbol_macro (symbol, handle, text, flags);
 
   return symbol;
 }
 
 m4_symbol *
-m4_macro_define (const char *name, lt_dlhandle handle, const char *text)
+m4_macro_define (const char *name, lt_dlhandle handle, const char *text,
+                int flags)
 {
   m4_symbol *symbol;
 
@@ -163,7 +169,7 @@ m4_macro_define (const char *name, lt_dl
   symbol = m4_symbol_define (name);
 
   if (symbol)
-    m4_symbol_macro (symbol, handle, text);
+    m4_symbol_macro (symbol, handle, text, flags);
 
   return symbol;
 }
@@ -175,5 +181,5 @@ m4_macro_table_install (lt_dlhandle hand
   const m4_macro *mp;
 
   for (mp = table; mp->name != NULL; mp++)
-    m4_macro_pushdef (mp->name, handle, mp->value);
+    m4_macro_pushdef (mp->name, handle, mp->value, 0);
 }
Index: m4/debug.c
===================================================================
RCS file: /cvsroot/m4/m4/m4/debug.c,v
retrieving revision 1.9
diff -u -p -u -r1.9 debug.c
--- m4/debug.c 2001/10/01 07:38:48 1.9
+++ m4/debug.c 2001/10/11 21:01:02
@@ -326,7 +326,7 @@ m4_trace_prepre (const char *name, int i
 /* Format the parts of a trace line, that can be made before the macro is
    actually expanded.  Used from expand_macro ().  */
 void
-m4_trace_pre (const char *name, int id, int argc, m4_token_data **argv)
+m4_trace_pre (const char *name, int id, int argc, m4_token **argv)
 {
   int i;
   const m4_builtin *bp;
@@ -343,14 +343,14 @@ m4_trace_pre (const char *name, int id, 
          if (i != 1)
            m4_trace_format (", ");
 
-         switch (M4_TOKEN_DATA_TYPE (argv[i]))
+         switch (TOKEN_TYPE (argv[i]))
            {
            case M4_TOKEN_TEXT:
              m4_trace_format ("%l%S%r", M4ARG (i));
              break;
 
            case M4_TOKEN_FUNC:
-             bp = m4_builtin_find_by_func (NULL, M4_TOKEN_DATA_FUNC(argv[i]));
+             bp = m4_builtin_find_by_func (NULL, TOKEN_FUNC (argv[i]));
              if (bp == NULL)
                {
                  M4ERROR ((warning_status, 0, _("\
@@ -360,7 +360,7 @@ INTERNAL ERROR: Builtin not found in bui
              m4_trace_format ("<%s>", bp->name);
              break;
 
-           default:
+           case M4_TOKEN_VOID:
              M4ERROR ((warning_status, 0, _("\
 INTERNAL ERROR: Bad token data type (m4_trace_pre ())")));
              abort ();
@@ -380,7 +380,7 @@ INTERNAL ERROR: Bad token data type (m4_
 /* Format the final part of a trace line and print it all.  Used from
    expand_macro ().  */
 void
-m4_trace_post (const char *name, int id, int argc, m4_token_data **argv,
+m4_trace_post (const char *name, int id, int argc, m4_token **argv,
            const char *expanded)
 {
   if (debug_level & M4_DEBUG_TRACE_CALL)
Index: m4/input.c
===================================================================
RCS file: /cvsroot/m4/m4/m4/input.c,v
retrieving revision 1.13
diff -u -p -u -r1.13 input.c
--- m4/input.c 2001/09/30 22:26:57 1.13
+++ m4/input.c 2001/10/11 21:01:04
@@ -142,7 +142,7 @@ static      int   file_peek                 (void);
 static int   file_read                 (void);
 static void  file_unget                (int ch);
 static void  file_clean                (void);
-static void  init_macro_token          (m4_token_data *td);
+static void  init_macro_token          (m4_token *td);
 static int   macro_peek                (void);
 static int   macro_read                (void);
 static int   match_input               (const unsigned char *s);
@@ -197,6 +197,7 @@ struct input_block
        {
          m4_builtin_func *func;/* pointer to macros function */
          lt_dlhandle handle;   /* originating module */
+         int flags;            /* flags associated with the builtin */
          boolean traced;       /* TRUE iff builtin is traced */
          boolean read;         /* TRUE iff block has been read */
        }
@@ -389,8 +390,7 @@ static struct input_funcs macro_funcs = 
 };
 
 void
-m4_push_macro (m4_builtin_func *func, lt_dlhandle handle,
-              boolean traced)
+m4_push_macro (m4_builtin_func *func, lt_dlhandle handle, int flags)
 {
   input_block *i;
 
@@ -404,10 +404,10 @@ m4_push_macro (m4_builtin_func *func, lt
                                     sizeof (struct input_block));
   i->funcs = &macro_funcs;
 
-  i->u.u_m.func = func;
-  i->u.u_m.handle = handle;
-  i->u.u_m.traced = traced;
-  i->u.u_m.read = FALSE;
+  i->u.u_m.func                = func;
+  i->u.u_m.handle      = handle;
+  i->u.u_m.flags       = flags;
+  i->u.u_m.read                = FALSE;
 
   i->prev = isp;
   isp = i;
@@ -594,7 +594,7 @@ m4_pop_wrapup (void)
 /* When a MACRO token is seen, next_token () uses init_macro_token
    to retrieve the value of the function pointer.  */
 static void
-init_macro_token (m4_token_data *td)
+init_macro_token (m4_token *td)
 {
   if (isp->funcs->read_func != macro_read)
     {
@@ -603,10 +603,10 @@ init_macro_token (m4_token_data *td)
       abort ();
     }
 
-  M4_TOKEN_DATA_TYPE (td) = M4_TOKEN_FUNC;
-  M4_TOKEN_DATA_FUNC (td) = isp->u.u_m.func;
-  M4_TOKEN_DATA_HANDLE (td) = isp->u.u_m.handle;
-  M4_TOKEN_TRACED (td) = isp->u.u_m.traced;
+  TOKEN_TYPE (td)      = M4_TOKEN_FUNC;
+  TOKEN_FUNC (td)      = isp->u.u_m.func;
+  TOKEN_HANDLE (td)    = isp->u.u_m.handle;
+  TOKEN_FLAGS (td)     = isp->u.u_m.flags;
 }
 
 
@@ -1009,7 +1009,7 @@ m4_set_syntax (char key, const unsigned 
    The storage pointed to by the fields in TD is therefore subject to
    change the next time next_token () is called.        */
 m4_token_t
-m4_next_token (m4_token_data *td)
+m4_next_token (m4_token *td)
 {
   int ch;
   int quote_level;
@@ -1200,8 +1200,10 @@ m4_next_token (m4_token_data *td)
 
   obstack_1grow (&token_stack, '\0');
 
-  M4_TOKEN_DATA_TYPE (td) = M4_TOKEN_TEXT;
-  M4_TOKEN_DATA_TEXT (td) = obstack_finish (&token_stack);
+  TOKEN_TYPE (td)      = M4_TOKEN_TEXT;
+  TOKEN_TEXT (td)      = obstack_finish (&token_stack);
+  TOKEN_HANDLE (td)    = NULL;
+  TOKEN_FLAGS (td)     = 0x0;
 
 #ifdef DEBUG_INPUT
   print_token("next_token", type, td);
@@ -1216,37 +1218,37 @@ m4_next_token (m4_token_data *td)
 static void  lex_debug (void);
 
 int
-m4_print_token (const char *s, m4_token_t t, m4_token_data *td)
+m4_print_token (const char *s, m4_token_t t, m4_token *td)
 {
   fprintf (stderr, "%s: ", s);
   switch (t)
     {                          /* TOKSW */
     case M4_TOKEN_SIMPLE:
-      fprintf (stderr, "char\t\"%s\"\n", M4_TOKEN_DATA_TEXT (td));
+      fprintf (stderr, "char\t\"%s\"\n",       TOKEN_TEXT (td));
       break;
 
     case M4_TOKEN_WORD:
-      fprintf (stderr, "word\t\"%s\"\n", M4_TOKEN_DATA_TEXT (td));
+      fprintf (stderr, "word\t\"%s\"\n",       TOKEN_TEXT (td));
       break;
 
     case M4_TOKEN_STRING:
-      fprintf (stderr, "string\t\"%s\"\n", M4_TOKEN_DATA_TEXT (td));
+      fprintf (stderr, "string\t\"%s\"\n",     TOKEN_TEXT (td));
       break;
 
     case M4_TOKEN_SPACE:
-      fprintf (stderr, "space\t\"%s\"\n", M4_TOKEN_DATA_TEXT (td));
+      fprintf (stderr, "space\t\"%s\"\n",      TOKEN_TEXT (td));
       break;
 
     case M4_TOKEN_MACDEF:
-      fprintf (stderr, "macro 0x%x\n", (int)M4_TOKEN_DATA_FUNC (td));
+      fprintf (stderr, "macro 0x%x\n",         (int) TOKEN_FUNC (td));
       break;
 
     case M4_TOKEN_EOF:
-      fprintf (stderr, "eof\n");
+      fprintf (stderr, "eof\n");
       break;
 
     case M4_TOKEN_NONE:
-      fprintf (stderr, "none\n");
+      fprintf (stderr, "none\n");
       break;
     }
   return 0;
@@ -1256,7 +1258,7 @@ static void
 lex_debug (void)
 {
   m4_token_t t;
-  m4_token_data td;
+  m4_token td;
 
   while ((t = next_token (&td)) != NULL)
     print_token ("lex", t, &td);
Index: m4/m4module.h
===================================================================
RCS file: /cvsroot/m4/m4/m4/m4module.h,v
retrieving revision 1.32
diff -u -p -u -r1.32 m4module.h
--- m4/m4module.h 2001/09/30 22:26:57 1.32
+++ m4/m4module.h 2001/10/11 21:01:06
@@ -30,11 +30,12 @@ BEGIN_C_DECLS
 
 /* Various declarations.  */
 
-typedef struct m4_symbol m4_symbol;
-typedef struct m4_module_data m4_module_data;
-typedef struct m4_token_data m4_token_data;
+typedef struct m4_module_data  m4_module_data;
+typedef struct m4_symbol       m4_symbol;
+typedef struct m4_token                m4_token;
 
-typedef void m4_builtin_func (struct obstack *, int, struct m4_token_data **);
+
+typedef void m4_builtin_func (struct obstack *, int, struct m4_token **);
 typedef void *m4_module_func (const char *);
 typedef int m4_symtab_apply_func (const char *name, m4_symbol *symbol, void 
*data);
 
@@ -59,6 +60,7 @@ typedef struct {
 extern void        m4_module_init   (void);
 extern lt_dlhandle  m4_module_load   (const char*, struct obstack*);
 extern void        m4_module_unload (const char*, struct obstack*);
+extern void        m4_module_unload_all (void);
 extern lt_dlhandle  m4_module_open   (const char*, struct obstack*);
 extern void        m4_module_close  (lt_dlhandle, struct obstack*);
 extern void        m4_module_close_all (struct obstack*);
@@ -71,18 +73,18 @@ extern lt_dlhandle  m4_module_find_by_bu
 
 
 extern m4_symbol *m4_macro_pushdef     (const char *name, lt_dlhandle handle,
-                                        const char *text);
+                                        const char *text, int flags);
 extern m4_symbol *m4_macro_define      (const char *name, lt_dlhandle handle,
-                                        const char *text);
+                                        const char *text, int flags);
 extern void      m4_macro_table_install (lt_dlhandle handle,
                                          const m4_macro *table);
 
 extern m4_symbol *m4_builtin_pushdef   (const char *name, lt_dlhandle handle,
-                                        const m4_builtin *bp);
+                                        m4_builtin_func *func, int flags);
 extern m4_symbol *m4_builtin_define    (const char *name, lt_dlhandle handle,
-                                        const m4_builtin *bp);
+                                        m4_builtin_func *func, int flags);
 extern void      m4_builtin_table_install (lt_dlhandle handle,
-                                           const m4_builtin *table);
+                                        const m4_builtin *table);
 
 extern const m4_builtin *m4_builtin_find_by_name (
                                const m4_builtin *, const char *);
@@ -102,9 +104,9 @@ extern m4_symbol *m4_symbol_define  (cons
 extern void       m4_symbol_popdef     (const char *);
 extern void       m4_symbol_delete     (const char *);
 extern void      m4_symbol_builtin     (m4_symbol *symbol, lt_dlhandle handle,
-                                        const m4_builtin *bp);
+                                        m4_builtin_func *func, int flags);
 extern void      m4_symbol_macro       (m4_symbol *symbol, lt_dlhandle handle,
-                                        const char *text);
+                                        const char *text, int flags);
 
 
 /* Various different token types.  */
@@ -123,25 +125,24 @@ typedef enum {
   M4_TOKEN_VOID,
   M4_TOKEN_TEXT,
   M4_TOKEN_FUNC
-} m4_token_data_t;
+} m4_data_t;
 
 typedef void m4_module_init_func   (lt_dlhandle, struct obstack*);
 typedef void m4_module_finish_func (lt_dlhandle, struct obstack*);
-
-extern m4_token_data_t  m4_token_data_type       (m4_token_data*);
-extern char           *m4_token_data_text        (m4_token_data*);
-extern m4_builtin_func *m4_token_data_func       (m4_token_data*);
 
+extern m4_token_t      m4_token_type     (m4_token *);
+extern char           *m4_token_text     (m4_token *);
+extern m4_builtin_func *m4_token_func    (m4_token *);
 
-#define M4ARG(i)       (argc > (i) ? m4_token_data_text (argv[i]) : "")
+#define M4ARG(i)       (argc > (i) ? m4_token_text (argv[i]) : "")
 
 #define M4BUILTIN(name)                                        \
   static void CONC(builtin_, name)                             \
-       (struct obstack *, int , m4_token_data **);
+       (struct obstack *, int , m4_token **);
 
 #define M4BUILTIN_HANDLER(name)                                \
   static void CONC(builtin_, name) (obs, argc, argv)           \
-       struct obstack *obs; int argc; m4_token_data **argv;
+       struct obstack *obs; int argc; m4_token **argv;
 
 #define M4INIT_HANDLER(name)                                   \
   void CONC(name, CONC(_LTX_, m4_init_module))                         \
@@ -193,12 +194,12 @@ m4_string ecomm;
 #define DEF_BCOMM "#"
 #define DEF_ECOMM "\n"
 
-boolean m4_bad_argc (m4_token_data *, int, int, int);
+boolean m4_bad_argc (m4_token *, int, int, int);
 const char *m4_skip_space (const char *);
-boolean m4_numeric_arg (m4_token_data *, const char *, int *);
+boolean m4_numeric_arg (m4_token *, const char *, int *);
 void m4_shipout_int (struct obstack *, int);
 void m4_shipout_string (struct obstack*, const char*, int, boolean);
-void m4_dump_args (struct obstack *obs, int argc, m4_token_data **argv, const 
char *sep, boolean quoted);
+void m4_dump_args (struct obstack *obs, int argc, m4_token **argv, const char 
*sep, boolean quoted);
 
 
 FILE *m4_debug;
@@ -299,8 +300,8 @@ boolean m4_debug_set_output (const char 
 void m4_debug_message_prefix (void);
 
 void m4_trace_prepre (const char *, int);
-void m4_trace_pre (const char *, int, int, m4_token_data **);
-void m4_trace_post (const char *, int, int, m4_token_data **,
+void m4_trace_pre (const char *, int, int, m4_token **);
+void m4_trace_post (const char *, int, int, m4_token **,
                           const char *);
 
 /* Exit code from last "syscmd" command.  */
@@ -309,8 +310,8 @@ int m4_expansion_level;
 
 const char *m4_expand_ranges (const char *s, struct obstack *obs);
 void m4_expand_input (void);
-void m4_call_macro (m4_symbol *, int, m4_token_data **, struct obstack *);
-void m4_process_macro (struct obstack *obs, m4_symbol *symbol, int argc, 
m4_token_data **argv);
+void m4_call_macro (m4_symbol *, int, m4_token **, struct obstack *);
+void m4_process_macro (struct obstack *obs, m4_symbol *symbol, int argc, 
m4_token **argv);
 
 
 
@@ -380,15 +381,14 @@ extern    void    m4_input_init   (void);
 extern void    m4_input_exit   (void);
 extern void    m4_syntax_init  (void);
 extern int     m4_peek_input   (void);
-extern m4_token_t m4_next_token (m4_token_data *);
+extern m4_token_t m4_next_token (m4_token *);
 extern void    m4_skip_line    (void);
 
 /* push back input */
 
 extern void    m4_push_file    (FILE *, const char *);
 extern void    m4_push_single  (int ch);
-extern void    m4_push_macro   (m4_builtin_func *, lt_dlhandle,
-                                          boolean);
+extern void    m4_push_macro   (m4_builtin_func *, lt_dlhandle, int);
 extern struct obstack *m4_push_string_init (void);
 extern const char *m4_push_string_finish (void);
 extern void    m4_push_wrapup  (const char *);
@@ -449,7 +449,7 @@ struct m4_dump_symbol_data
 };
 
 extern int m4_dump_symbol (const char *name, m4_symbol *symbol, void *data);
-extern void m4_dump_symbols (struct m4_dump_symbol_data *data, int argc, 
m4_token_data **argv, boolean complain);
+extern void m4_dump_symbols (struct m4_dump_symbol_data *data, int argc, 
m4_token **argv, boolean complain);
 
 
 
Index: m4/m4private.h
===================================================================
RCS file: /cvsroot/m4/m4/m4/m4private.h,v
retrieving revision 1.12
diff -u -p -u -r1.12 m4private.h
--- m4/m4private.h 2001/09/30 22:26:57 1.12
+++ m4/m4private.h 2001/10/11 21:01:06
@@ -33,47 +33,58 @@ struct m4_module_data {
 };
 
 
-struct m4_token_data {
-  m4_token_data        *       next;
-  m4_token_data_t      type;
-  boolean              traced;
-  boolean              macro_args;
-  boolean              blind_no_args;
+/* m4_token.flags bit masks:  */
+
+#define TOKEN_MACRO_ARGS_BIT           (1 << 0)
+#define TOKEN_BLIND_ARGS_BIT           (1 << 1)
+
+
+struct m4_token {
+  m4_token *   next;
   lt_dlhandle          handle;
+  int                  flags;
+
+  m4_data_t            type;
   union {
     char *             text;
     m4_builtin_func *  func;
   } u;
 };
+
+#define TOKEN_NEXT(T)          ((T)->next)
+#define TOKEN_HANDLE(T)        ((T)->handle)
+#define TOKEN_FLAGS(T)         ((T)->flags)
+#define TOKEN_TYPE(T)          ((T)->type)
+#define TOKEN_TEXT(T)          ((T)->u.text)
+#define TOKEN_FUNC(T)          ((T)->u.func)
 
-#define M4_TOKEN_DATA_NEXT(Td)         ((Td)->next)
-#define M4_TOKEN_DATA_TYPE(Td)         ((Td)->type)
-#define M4_TOKEN_TRACED(Td)            ((Td)->traced)
-#define M4_TOKEN_MACRO_ARGS(Td)                ((Td)->macro_args)
-#define M4_TOKEN_BLIND_NO_ARGS(Td)     ((Td)->blind_no_args)
-#define M4_TOKEN_DATA_HANDLE(Td)       ((Td)->handle)
-#define M4_TOKEN_DATA_TEXT(Td)         ((Td)->u.text)
-#define M4_TOKEN_DATA_FUNC(Td)         ((Td)->u.func)
+#define BIT_TEST(flags, bit)   (((flags) & (bit)) == (bit))
+#define BIT_SET(flags, bit)    ((flags) |= (bit))
+#define BIT_RESET(flags, bit)  ((flags) &= ~(bit))
 
+
 /* Redefine the exported function to this faster
    macro based version for internal use by the m4 code. */
 #undef M4ARG
-#define M4ARG(i)       (argc > (i) ? M4_TOKEN_DATA_TEXT (argv[i]) : "")
+#define M4ARG(i)       (argc > (i) ? TOKEN_TEXT (argv[i]) : "")
+
 
 struct m4_symbol
 {
-  m4_token_data *data;
+  boolean      traced;
+  m4_token *   token;
 };
+
+#define SYMBOL_TRACED(S)       ((S)->traced)
+#define SYMBOL_TOKEN(S)                ((S)->token)
+
+#define SYMBOL_NEXT(S)         (TOKEN_NEXT   (SYMBOL_TOKEN (S)))
+#define SYMBOL_HANDLE(S)       (TOKEN_HANDLE (SYMBOL_TOKEN (S)))
+#define SYMBOL_FLAGS(S)                (TOKEN_FLAGS  (SYMBOL_TOKEN (S)))
+#define SYMBOL_TYPE(S)         (TOKEN_TYPE   (SYMBOL_TOKEN (S)))
+#define SYMBOL_TEXT(S)         (TOKEN_TEXT   (SYMBOL_TOKEN (S)))
+#define SYMBOL_FUNC(S)         (TOKEN_FUNC   (SYMBOL_TOKEN (S)))
 
-#define M4_SYMBOL_DATA(S)         ((S)->data)
-#define M4_SYMBOL_DATA_NEXT(S)    (M4_TOKEN_DATA_NEXT (M4_SYMBOL_DATA(S)))
-#define M4_SYMBOL_TYPE(S)         (M4_TOKEN_DATA_TYPE (M4_SYMBOL_DATA(S)))
-#define M4_SYMBOL_TRACED(S)       (M4_TOKEN_TRACED (M4_SYMBOL_DATA(S)))
-#define M4_SYMBOL_MACRO_ARGS(S)           (M4_TOKEN_MACRO_ARGS 
(M4_SYMBOL_DATA(S)))
-#define M4_SYMBOL_BLIND_NO_ARGS(S) (M4_TOKEN_BLIND_NO_ARGS (M4_SYMBOL_DATA(S)))
-#define M4_SYMBOL_TEXT(S)         (M4_TOKEN_DATA_TEXT (M4_SYMBOL_DATA(S)))
-#define M4_SYMBOL_FUNC(S)         (M4_TOKEN_DATA_FUNC (M4_SYMBOL_DATA(S)))
-#define M4_SYMBOL_HANDLE(S)       (M4_TOKEN_DATA_HANDLE (M4_SYMBOL_DATA(S)))
 
 
 /* Debugging the memory allocator.  */
Index: m4/macro.c
===================================================================
RCS file: /cvsroot/m4/m4/m4/macro.c,v
retrieving revision 1.15
diff -u -p -u -r1.15 macro.c
--- m4/macro.c 2001/10/01 07:38:48 1.15
+++ m4/macro.c 2001/10/11 21:01:07
@@ -24,7 +24,7 @@
 #include "m4private.h"
 
 static void expand_macro (const char *name, m4_symbol *);
-static void expand_token (struct obstack *, m4_token_t, m4_token_data *);
+static void expand_token (struct obstack *, m4_token_t, m4_token *);
 
 /* Current recursion level in expand_macro ().  */
 int m4_expansion_level = 0;
@@ -37,7 +37,7 @@ void
 m4_expand_input (void)
 {
   m4_token_t t;
-  m4_token_data td;
+  m4_token td;
 
   while ((t = m4_next_token (&td)) != M4_TOKEN_EOF)
     expand_token ((struct obstack *) NULL, t, &td);
@@ -49,10 +49,10 @@ m4_expand_input (void)
    macro definition.  If they have, they are expanded as macros, otherwise
    the text are just copied to the output.  */
 static void
-expand_token (struct obstack *obs, m4_token_t t, m4_token_data *td)
+expand_token (struct obstack *obs, m4_token_t t, m4_token *td)
 {
   m4_symbol *symbol;
-  char *text = xstrdup (M4_TOKEN_DATA_TEXT (td));
+  char *text = xstrdup (TOKEN_TEXT (td));
 
   switch (t)
     {                          /* TOKSW */
@@ -74,13 +74,13 @@ expand_token (struct obstack *obs, m4_to
          ++p;
 
        symbol = m4_symbol_lookup (p);
-       if (symbol == NULL || M4_SYMBOL_TYPE (symbol) == M4_TOKEN_VOID
-           || (M4_SYMBOL_TYPE (symbol) == M4_TOKEN_FUNC
-               && M4_SYMBOL_BLIND_NO_ARGS (symbol)
+       if (symbol == NULL || SYMBOL_TYPE (symbol) == M4_TOKEN_VOID
+           || (SYMBOL_TYPE (symbol) == M4_TOKEN_FUNC
+               && BIT_TEST (SYMBOL_FLAGS (symbol), TOKEN_BLIND_ARGS_BIT)
                && !M4_IS_OPEN(m4_peek_input ())))
          {
-           m4_shipout_text (obs, M4_TOKEN_DATA_TEXT (td),
-                            strlen (M4_TOKEN_DATA_TEXT (td)));
+           m4_shipout_text (obs, TOKEN_TEXT (td),
+                            strlen (TOKEN_TEXT (td)));
          }
        else
          expand_macro (p, symbol);
@@ -105,14 +105,14 @@ expand_token (struct obstack *obs, m4_to
    the last for the active macro call.  The argument are build on the
    obstack OBS, indirectly through expand_token ().     */
 static boolean
-expand_argument (struct obstack *obs, m4_token_data *argp)
+expand_argument (struct obstack *obs, m4_token *argp)
 {
   m4_token_t t;
-  m4_token_data td;
+  m4_token td;
   char *text;
   int paren_level;
 
-  M4_TOKEN_DATA_TYPE (argp) = M4_TOKEN_VOID;
+  TOKEN_TYPE (argp) = M4_TOKEN_VOID;
 
   /* Skip leading white space.  */
   do
@@ -129,7 +129,7 @@ expand_argument (struct obstack *obs, m4
       switch (t)
        {                       /* TOKSW */
        case M4_TOKEN_SIMPLE:
-         text = M4_TOKEN_DATA_TEXT (&td);
+         text = TOKEN_TEXT (&td);
          if ((M4_IS_COMMA(*text) || M4_IS_CLOSE(*text)) && paren_level == 0)
            {
 
@@ -137,12 +137,12 @@ expand_argument (struct obstack *obs, m4
              obstack_1grow (obs, '\0');
              text = obstack_finish (obs);
 
-             if (M4_TOKEN_DATA_TYPE (argp) == M4_TOKEN_VOID)
+             if (TOKEN_TYPE (argp) == M4_TOKEN_VOID)
                {
-                 M4_TOKEN_DATA_TYPE (argp) = M4_TOKEN_TEXT;
-                 M4_TOKEN_DATA_TEXT (argp) = text;
+                 TOKEN_TYPE (argp) = M4_TOKEN_TEXT;
+                 TOKEN_TEXT (argp) = text;
                }
-             return (boolean) (M4_IS_COMMA(*M4_TOKEN_DATA_TEXT (&td)));
+             return (boolean) (M4_IS_COMMA(*TOKEN_TEXT (&td)));
            }
 
          if (M4_IS_OPEN(*text))
@@ -166,10 +166,10 @@ expand_argument (struct obstack *obs, m4
        case M4_TOKEN_MACDEF:
          if (obstack_object_size (obs) == 0)
            {
-             M4_TOKEN_DATA_TYPE (argp)   = M4_TOKEN_FUNC;
-             M4_TOKEN_DATA_HANDLE (argp) = M4_TOKEN_DATA_HANDLE (&td);
-             M4_TOKEN_DATA_FUNC (argp)   = M4_TOKEN_DATA_FUNC (&td);
-             M4_TOKEN_TRACED (argp)      = M4_TOKEN_TRACED (&td);
+             TOKEN_TYPE (argp)   = M4_TOKEN_FUNC;
+             TOKEN_FUNC (argp)   = TOKEN_FUNC (&td);
+             TOKEN_HANDLE (argp) = TOKEN_HANDLE (&td);
+             TOKEN_FLAGS (argp)  = TOKEN_FLAGS (&td);
            }
          break;
 
@@ -191,14 +191,16 @@ collect_arguments (const char *name, m4_
                   struct obstack *argptr, struct obstack *arguments)
 {
   int ch;                      /* lookahead for ( */
-  m4_token_data td;
-  m4_token_data *tdp;
+  m4_token td;
+  m4_token *tdp;
   boolean more_args;
-  boolean groks_macro_args = M4_SYMBOL_MACRO_ARGS (symbol);
+  boolean groks_macro_args;
 
-  M4_TOKEN_DATA_TYPE (&td) = M4_TOKEN_TEXT;
-  M4_TOKEN_DATA_TEXT (&td) = (char *) name;
-  tdp = (m4_token_data *) obstack_copy (arguments, (void *) &td, sizeof (td));
+  groks_macro_args = BIT_TEST (SYMBOL_FLAGS (symbol), TOKEN_MACRO_ARGS_BIT);
+
+  TOKEN_TYPE (&td) = M4_TOKEN_TEXT;
+  TOKEN_TEXT (&td) = (char *) name;
+  tdp = (m4_token *) obstack_copy (arguments, (void *) &td, sizeof (td));
   obstack_grow (argptr, (void *) &tdp, sizeof (tdp));
 
   ch = m4_peek_input ();
@@ -209,12 +211,12 @@ collect_arguments (const char *name, m4_
        {
          more_args = expand_argument (arguments, &td);
 
-         if (!groks_macro_args && M4_TOKEN_DATA_TYPE (&td) == M4_TOKEN_FUNC)
+         if (!groks_macro_args && TOKEN_TYPE (&td) == M4_TOKEN_FUNC)
            {
-             M4_TOKEN_DATA_TYPE (&td) = M4_TOKEN_TEXT;
-             M4_TOKEN_DATA_TEXT (&td) = "";
+             TOKEN_TYPE (&td) = M4_TOKEN_TEXT;
+             TOKEN_TEXT (&td) = "";
            }
-         tdp = (m4_token_data *)
+         tdp = (m4_token *)
            obstack_copy (arguments, (void *) &td, sizeof (td));
          obstack_grow (argptr, (void *) &tdp, sizeof (tdp));
        }
@@ -231,20 +233,20 @@ collect_arguments (const char *name, m4_
    arguments to the call, stored in the ARGV table.  The expansion is
    left on the obstack EXPANSION.  Macro tracing is also handled here.  */
 void
-m4_call_macro (m4_symbol *symbol, int argc, m4_token_data **argv,
+m4_call_macro (m4_symbol *symbol, int argc, m4_token **argv,
               struct obstack *expansion)
 {
-  switch (M4_SYMBOL_TYPE (symbol))
+  switch (SYMBOL_TYPE (symbol))
     {
     case M4_TOKEN_FUNC:
-      (*M4_SYMBOL_FUNC (symbol)) (expansion, argc, argv);
+      (*SYMBOL_FUNC (symbol)) (expansion, argc, argv);
       break;
 
     case M4_TOKEN_TEXT:
       m4_process_macro (expansion, symbol, argc, argv);
       break;
 
-    default:
+    case M4_TOKEN_VOID:
       M4ERROR ((warning_status, 0,
                _("INTERNAL ERROR: Bad symbol type in call_macro ()")));
       abort ();
@@ -263,7 +265,7 @@ expand_macro (const char *name, m4_symbo
 {
   struct obstack arguments;
   struct obstack argptr;
-  m4_token_data **argv;
+  m4_token **argv;
   int argc;
   struct obstack *expansion;
   const char *expanded;
@@ -279,7 +281,7 @@ ERROR: Recursion limit of %d exceeded, u
   macro_call_id++;
   my_call_id = macro_call_id;
 
-  traced = (boolean) ((debug_level & M4_DEBUG_TRACE_ALL) || M4_SYMBOL_TRACED 
(symbol));
+  traced = (boolean) ((debug_level & M4_DEBUG_TRACE_ALL) || SYMBOL_TRACED 
(symbol));
 
   obstack_init (&argptr);
   obstack_init (&arguments);
@@ -289,8 +291,8 @@ ERROR: Recursion limit of %d exceeded, u
 
   collect_arguments (name, symbol, &argptr, &arguments);
 
-  argc = obstack_object_size (&argptr) / sizeof (m4_token_data *);
-  argv = (m4_token_data **) obstack_finish (&argptr);
+  argc = obstack_object_size (&argptr) / sizeof (m4_token *);
+  argv = (m4_token **) obstack_finish (&argptr);
 
   if (traced)
     m4_trace_pre (name, my_call_id, argc, argv);
@@ -315,12 +317,12 @@ ERROR: Recursion limit of %d exceeded, u
    as usual.  */
 void
 m4_process_macro (struct obstack *obs, m4_symbol *symbol, int argc,
-                 m4_token_data **argv)
+                 m4_token **argv)
 {
   const unsigned char *text;
   int i;
 
-  for (text = M4_SYMBOL_TEXT (symbol); *text != '\0';)
+  for (text = SYMBOL_TEXT (symbol); *text != '\0';)
     {
       if (*text != '$')
        {
Index: m4/module.c
===================================================================
RCS file: /cvsroot/m4/m4/m4/module.c,v
retrieving revision 1.14
diff -u -p -u -r1.14 module.c
--- m4/module.c 2001/09/30 22:26:57 1.14
+++ m4/module.c 2001/10/11 21:01:08
@@ -59,14 +59,14 @@
  * names to the expansion text.  Any macros defined in `m4_macro_table'
  * are installed into the M4 symbol table with m4_macro_table_install().
  *
- * Each time a module is loaded, the module function 
+ * Each time a module is loaded, the module function
  * "void m4_init_module (lt_dlhandle handle, struct obstack *obs)" is
  * called, if defined.  Any value stored in OBS by this function becomes
- * the expansion of the macro which called it.  Before M4 exits, all 
+ * the expansion of the macro which called it.  Before M4 exits, all
  * modules are unloaded and the function
  * "void m4_finish_module (lt_dlhandle handle, struct obstack *obs)" is
  * called, if defined.  It is safe to load the same module several times:
- * the init and finish functions will also be called multiple times in 
+ * the init and finish functions will also be called multiple times in
  * this case.
  *
  * To unload a module, use m4_module_unload(). which uses
@@ -169,7 +169,7 @@ void
 m4_module_init (void)
 {
   int errors = 0;
-  
+
   /* Do this only once!  If we already have a caller_id, then the
      module system has already been initialised.  */
   if (m4_caller_id)
@@ -205,7 +205,7 @@ m4_module_init (void)
          ++errors;
        }
     }
-  
+
   if (!errors)
     errors = lt_dlsetsearchpath (MODULE_PATH);
 
@@ -294,7 +294,7 @@ m4_module_open (const char *name, struct
   if (handle)
     {
       const lt_dlinfo  *info   = lt_dlgetinfo (handle);
-      
+
       if (info && (info->ref_count == 1))
        {
          m4_module_data *data  = XMALLOC (m4_module_data, 1);
@@ -308,7 +308,7 @@ m4_module_open (const char *name, struct
          if (stale)
            {
              xfree (stale);
-         
+
              M4ERROR ((warning_status, 0,
                        _("Warning: overiding stale caller data in module 
`%s'"),
                        name));
@@ -344,7 +344,7 @@ m4_module_close (lt_dlhandle handle, str
       M4_DEBUG_MESSAGE1("module %s: finish hook called", name);
 #endif /* DEBUG_MODULES */
     }
-      
+
   if (!lt_dlisresident (handle))
     {
       const lt_dlinfo  *info   = lt_dlgetinfo (handle);
@@ -377,7 +377,7 @@ m4_module_close (lt_dlhandle handle, str
                _("ERROR: cannot close module `%s': %s"),
                name, m4_module_dlerror ()));
     }
-  
+
   xfree (name);
 }
 
Index: m4/symtab.c
===================================================================
RCS file: /cvsroot/m4/m4/m4/symtab.c,v
retrieving revision 1.24
diff -u -p -u -r1.24 symtab.c
--- m4/symtab.c 2001/09/30 22:26:57 1.24
+++ m4/symtab.c 2001/10/11 21:01:09
@@ -27,47 +27,136 @@
    symbol table is an abstract hash table type implemented in hash.c.  Each
    symbol is represented by `struct m4_symbol', which is stored in the hash
    table keyed by the symbol name.  As a special case, to facilitate the
-   "pushdef" and "popdef" builtins, the value stored against each key us a
+   "pushdef" and "popdef" builtins, the value stored against each key is a
    stack of `struct m4_symbol'. All the value entries for a symbol name are
    simply ordered on the stack by age.  The most recently pushed definition
-   will then always be the first found.  */
+   will then always be the first found.
 
+   Also worthy of mention is the way traced symbols are managed:  the trace
+   bit is associated with a particular symbol name.  If a symbol is
+   undefined and then redefined, it does not lose its trace bit (in GNU
+   mode).  This is achieved by not removing traced symbol names from the
+   symbol table, even if their value stack is empty.  That way, when the
+   name is given a new value, it is pushed onto the empty stack, and the
+   trace bit attached to the name was never lost.  There is a small amount
+   of fluff in these functions to make sure that such symbols (with empty
+   value stacks) are invisible to the users of this module.  */
+
 static size_t  m4_symtab_hash          (const void *key);
 static int     m4_symtab_cmp           (const void *key, const void *try);
-static void    m4_token_data_delete    (m4_token_data *data);
-static void    m4_symbol_pop           (m4_symbol *symbol);
-static void    m4_symbol_del           (m4_symbol *symbol);
 static int     m4_symbol_destroy       (const char *name, m4_symbol *symbol,
                                         void *data);
 
-
 /* Pointer to symbol table.  */
 m4_hash *m4_symtab = 0;
 
 
+
 
+/* -- SYMBOL TABLE MANAGEMENT --
+
+   These functions are used to manage a symbol table as a whole.  */
+
 void
 m4_symtab_init (void)
 {
   m4_symtab = m4_hash_new (m4_symtab_hash, m4_symtab_cmp);
 }
 
+/* The following function is used for the cases where we want to do
+   something to each and every symbol in the table.  This function
+   traverses the symbol table, and calls a specified function FUNC for
+   each symbol in the table.  FUNC is called with a pointer to the
+   symbol, and the DATA argument.  */
 int
-m4_symbol_destroy (const char *name, m4_symbol *symbol, void *data)
+m4_symtab_apply (m4_symtab_apply_func *func, void *data)
 {
-  m4_symbol_delete (name);
-  return 0;
+  int result = 0;
+  m4_hash_iterator *place = 0;
+
+  assert (func);
+
+  while ((place = m4_hash_iterator_next (m4_symtab, place)))
+    {
+      const char *name = (const char *) m4_hash_iterator_key (place);
+      m4_symbol *symbol = (m4_symbol *) m4_hash_iterator_value (place);
+      result = (*func) (name, symbol, data);
+
+      if (result != 0)
+       break;
+    }
+
+  return result;
+}
+
+
+/* Remove every symbol that references the given module handle from
+   the symbol table.  */
+void
+m4_symtab_remove_module_references (lt_dlhandle handle)
+{
+  m4_hash_iterator *place = 0;
+
+  assert (handle);
+
+   /* Traverse each symbol name in the hash table.  */
+  while ((place = m4_hash_iterator_next (m4_symtab, place)))
+    {
+      m4_symbol *symbol = (m4_symbol *) m4_hash_iterator_value (place);
+      m4_token *data = SYMBOL_TOKEN (symbol);
+
+      /* For symbols that have token data... */
+      if (data)
+       {
+         /* Purge any shadowed references.  */
+         while (TOKEN_NEXT (data))
+           {
+             m4_token *next = TOKEN_NEXT (data);
+
+             if (TOKEN_HANDLE (next) == handle)
+               {
+                 TOKEN_NEXT (data) = TOKEN_NEXT (next);
+
+                 if (TOKEN_TYPE (next) == M4_TOKEN_TEXT)
+                   XFREE (TOKEN_TEXT (next));
+                 XFREE (next);
+               }
+             else
+               data = next;
+           }
+
+         /* Purge the live reference if necessary.  */
+         if (SYMBOL_HANDLE (symbol) == handle)
+           m4_symbol_popdef (m4_hash_iterator_key (place));
+       }
+    }
 }
 
+/* Release all of the memory used by the symbol table.  */
 void
 m4_symtab_exit (void)
 {
   m4_symtab_apply (m4_symbol_destroy, NULL);
-  m4_hash_delete (m4_symtab);
-  m4_hash_exit ();
+  m4_hash_delete  (m4_symtab);
+  m4_hash_exit    ();
 }
 
-/* Return a hashvalue for a string, from GNU-emacs.  */
+/* This callback is used exclusively by m4_symtab_exit(), to cleanup
+   the memory used by the symbol table.  As such, the trace bit is reset
+   on every symbol so that m4_symbol_popdef() doesn't try to preserve
+   the table entry.  */
+static int
+m4_symbol_destroy (const char *name, m4_symbol *symbol, void *data)
+{
+  SYMBOL_TRACED (symbol) = FALSE;
+
+  while (m4_hash_lookup (m4_symtab, name))
+    m4_symbol_popdef (name);
+
+  return 0;
+}
+
+/* Return a hash value for a string, from GNU Emacs.  */
 static size_t
 m4_symtab_hash (const void *key)
 {
@@ -85,6 +174,8 @@ m4_symtab_hash (const void *key)
   return val;
 }
 
+/* Comparison function for hash keys -- used by the underlying
+   hash table ADT when searching for a key match during name lookup.  */
 static int
 m4_symtab_cmp (const void *key, const void *try)
 {
@@ -92,89 +183,46 @@ m4_symtab_cmp (const void *key, const vo
 }
 
 
-
-static void
-m4_token_data_delete (m4_token_data *data)
-{
-  assert (data);
-  assert (M4_TOKEN_DATA_NEXT (data) == 0);
-
-  if (M4_TOKEN_DATA_TYPE (data) == M4_TOKEN_TEXT)
-    XFREE (M4_TOKEN_DATA_TEXT (data));
-
-  XFREE (data);
-}
-
-void
-m4_symbol_pop (m4_symbol *symbol)
-{
-  m4_token_data *stale;
-
-  assert (symbol);
-  assert (M4_SYMBOL_DATA_NEXT (symbol));
 
-  stale                                = M4_SYMBOL_DATA (symbol);
-  M4_SYMBOL_DATA (symbol)      = M4_TOKEN_DATA_NEXT (stale);
-
-#ifndef NDEBUG
-  M4_TOKEN_DATA_NEXT (stale) = 0;
-#endif
-  m4_token_data_delete (stale);
-}
-
-/* Free all storage associated with a symbol.  */
-void
-m4_symbol_del (m4_symbol *symbol)
-{
-  assert (symbol);
-
-  while (M4_SYMBOL_DATA_NEXT (symbol))
-    m4_symbol_pop (symbol);
-
-  assert (M4_SYMBOL_DATA_NEXT (symbol) == 0);
-
-  if (M4_SYMBOL_TYPE (symbol) == M4_TOKEN_TEXT)
-    XFREE (M4_SYMBOL_TEXT (symbol));
-
-  XFREE (M4_SYMBOL_DATA (symbol));
-  XFREE (symbol);
-}
+
+/* -- SYMBOL MANAGEMENT --
 
+   The following functions manipulate individual symbols within
+   an existing table.  */
 
-
 /* Return the symbol associated to NAME, or else NULL.  */
 m4_symbol *
 m4_symbol_lookup (const char *name)
 {
   m4_symbol **psymbol = (m4_symbol **) m4_hash_lookup (m4_symtab, name);
 
-  /* If just searching, return status of search.  */
-  return psymbol ? *psymbol : 0;
+  /* If just searching, return status of search -- if only an empty
+     struct is returned, that is treated as a failed lookup.  */
+  return (psymbol && SYMBOL_TOKEN (*psymbol)) ? *psymbol : 0;
 }
 
 
-/* Push a new slot associted to NAME, and return it.  */
+/* Insert NAME into the symbol table.  If there is already a symbol
+   associated with NAME, push the new value on top of the value stack
+   for this symbol.  Otherwise create a new association.  */
 m4_symbol *
 m4_symbol_pushdef (const char *name)
 {
   m4_symbol **psymbol = (m4_symbol **) m4_hash_lookup (m4_symtab, name);
 
-  /* Insert a name in the symbol table.  If there is already a symbol
-     with the name, push the new value on top of the value stack for
-     this symbol.  */
-  m4_symbol *symbol    = 0;
-  m4_token_data *value = XCALLOC (m4_token_data, 1);
+  m4_symbol *symbol = 0;
+  m4_token *value   = XCALLOC (m4_token, 1);
 
   if (psymbol)
     {
       symbol = *psymbol;
-      M4_TOKEN_DATA_NEXT (value) = M4_SYMBOL_DATA (symbol);
+      TOKEN_NEXT (value) = SYMBOL_TOKEN (symbol);
     }
   else
     symbol = XCALLOC (m4_symbol, 1);
 
-  M4_SYMBOL_DATA (symbol)      = value;
-  M4_SYMBOL_TYPE (symbol)      = M4_TOKEN_VOID;
+  SYMBOL_TOKEN (symbol)        = value;
+  SYMBOL_TYPE (symbol) = M4_TOKEN_VOID;
 
   if (!psymbol)
     m4_hash_insert (m4_symtab, xstrdup (name), symbol);
@@ -183,181 +231,165 @@ m4_symbol_pushdef (const char *name)
 }
 
 
-/* Return a slot associated to NAME, creating it if needed.  */
+/* Return the symbol associated with NAME in the symbol table, creating
+   a new symbol if necessary.  */
 m4_symbol *
 m4_symbol_define (const char *name)
 {
-  m4_symbol **psymbol = (m4_symbol **) m4_hash_lookup (m4_symtab, name);
-  if (psymbol)
-    return *psymbol;
-  else
-    return m4_symbol_pushdef (name);
+  m4_symbol *symbol = m4_symbol_lookup (name);
+
+  if (symbol)
+    return symbol;
+
+  return m4_symbol_pushdef (name);
 }
 
 
-/* Remove the topmost definition associated to NAME.  */
+/* Pop the topmost value stack entry from the symbol associated with
+   NAME, deleting it from the table entirely if that was the last
+   remaining value in the stack.  */
 void
 m4_symbol_popdef (const char *name)
 {
-  m4_symbol **psymbol = (m4_symbol **) m4_hash_lookup (m4_symtab, name);
+  m4_symbol **psymbol  = (m4_symbol **) m4_hash_lookup (m4_symtab, name);
+  m4_token  *stale     = NULL;
 
   assert (psymbol);
+  assert (*psymbol);
 
-  if (M4_SYMBOL_DATA_NEXT (*psymbol))
-    m4_symbol_pop (*psymbol);
-  else
+  stale = SYMBOL_TOKEN (*psymbol);
+
+  if (stale)
     {
-      xfree (m4_hash_remove (m4_symtab, name));
-      m4_symbol_del (*psymbol);
+      SYMBOL_TOKEN (*psymbol) = TOKEN_NEXT (stale);
+
+      if (TOKEN_TYPE (stale) == M4_TOKEN_TEXT)
+       XFREE (TOKEN_TEXT (stale));
+      XFREE (stale);
     }
+
+  /* Only remove the hash table entry if the last value in the
+     symbol value stack was successfully removed.  */
+  if (!SYMBOL_TOKEN (*psymbol))
+    if (no_gnu_extensions || !SYMBOL_TRACED (*psymbol))
+      {
+       XFREE (*psymbol);
+       xfree (m4_hash_remove (m4_symtab, name));
+      }
 }
 
 
-/* Remove all the definitions associated with NAME.  */
+/* Pop all values from the symbol associated with NAME.  */
 void
 m4_symbol_delete (const char *name)
 {
-  m4_symbol **psymbol = (m4_symbol **) m4_hash_lookup (m4_symtab, name);
-
-  assert (psymbol);
-
-#ifdef DEBUG_SYM
-  M4_DEBUG_MESSAGE1("symbol %s recycled.", name);
-#endif
-
-  xfree (m4_hash_remove (m4_symtab, name));
-  m4_symbol_del (*psymbol);
+  while (m4_symbol_lookup (name))
+    m4_symbol_popdef (name);
 }
 
 
+
+/* Set the type and value of a symbol according to the passed
+   arguments.  This function is usually passed a newly pushdef()d symbol
+   that is already interned in the symbol table.  The traced bit should
+   be appropriately set by the caller.  */
 void
 m4_symbol_builtin (m4_symbol *symbol, lt_dlhandle handle,
-                  const m4_builtin *bp)
+                  m4_builtin_func *func, int flags)
 {
   assert (symbol);
   assert (handle);
-  assert (bp);
+  assert (func);
 
-  if (M4_SYMBOL_TYPE (symbol) == M4_TOKEN_TEXT)
-    xfree (M4_SYMBOL_TEXT (symbol));
+  if (SYMBOL_TYPE (symbol) == M4_TOKEN_TEXT)
+    xfree (SYMBOL_TEXT (symbol));
 
-  M4_SYMBOL_HANDLE (symbol)            = handle;
-  M4_SYMBOL_TYPE (symbol)              = M4_TOKEN_FUNC;
-  M4_SYMBOL_MACRO_ARGS (symbol)                = bp->groks_macro_args;
-  M4_SYMBOL_BLIND_NO_ARGS (symbol)     = bp->blind_if_no_args;
-  M4_SYMBOL_FUNC (symbol)              = bp->func;
-  /* Do not reset M4_SYMBOL_TRACED as it means that --trace would be
-     usable only for existing macros.  m4_symbol_lookup takes care
-     of its proper initialisation.  */
+  SYMBOL_TYPE (symbol)         = M4_TOKEN_FUNC;
+  SYMBOL_FUNC (symbol)         = func;
+  SYMBOL_HANDLE (symbol)       = handle;
+  SYMBOL_FLAGS (symbol)                = flags;
 }
 
+/* ...and similarly for macro valued symbols.  */
 void
-m4_symbol_macro (m4_symbol *symbol, lt_dlhandle handle, const char *text)
+m4_symbol_macro (m4_symbol *symbol, lt_dlhandle handle,
+                const char *text, int flags)
 {
   assert (symbol);
 
-  if (M4_SYMBOL_TYPE (symbol) == M4_TOKEN_TEXT)
-    xfree (M4_SYMBOL_TEXT (symbol));
+  if (SYMBOL_TYPE (symbol) == M4_TOKEN_TEXT)
+    xfree (SYMBOL_TEXT (symbol));
 
-  M4_SYMBOL_HANDLE (symbol)            = handle;
-  M4_SYMBOL_TYPE (symbol)              = M4_TOKEN_TEXT;
-  M4_SYMBOL_MACRO_ARGS (symbol)                = FALSE;
-  M4_SYMBOL_BLIND_NO_ARGS (symbol)     = FALSE;
-  M4_SYMBOL_TEXT (symbol)              = xstrdup (text);
-  /* Do not reset M4_SYMBOL_TRACED as it means that --trace would be
-     usable only for existing macros.  m4_symbol_lookup takes care
-     of its proper initialisation.  */
+  SYMBOL_TYPE (symbol)                 = M4_TOKEN_TEXT;
+  SYMBOL_TEXT (symbol)                 = xstrdup (text);
+  SYMBOL_HANDLE (symbol)       = handle;
+  SYMBOL_FLAGS (symbol)                = 0;
 }
 
-/* Remove every symbol that references the given module handle from
-   the symbol table.  */
-void
-m4_symtab_remove_module_references (lt_dlhandle handle)
-{
-  m4_hash_iterator *place = 0;
-
-  assert (handle);
 
-   /* Traverse each symbol name in the hash table.  */
-  while ((place = m4_hash_iterator_next (m4_symtab, place)))
-    {
-      m4_symbol *symbol = (m4_symbol *) m4_hash_iterator_value (place);
-      m4_token_data *data = M4_SYMBOL_DATA (symbol);
 
-      /* Purge any shadowed references.  */
-      while (M4_TOKEN_DATA_NEXT (data))
-       {
-         m4_token_data *next = M4_TOKEN_DATA_NEXT (data);
-
-         if (M4_TOKEN_DATA_HANDLE (next) == handle)
-           {
-             M4_TOKEN_DATA_NEXT (data) = M4_TOKEN_DATA_NEXT (next);
-             m4_token_data_delete (next);
-           }
-         else
-           data = next;
-       }
+
+#ifdef DEBUG_SYM
 
-      /* Purge live reference.  */
-      if (M4_SYMBOL_HANDLE (symbol) == handle)
-       {
-         if (M4_SYMBOL_DATA_NEXT (symbol))
-           m4_symbol_pop (symbol);
-         else
-           {
-             xfree (m4_hash_remove (m4_symtab, m4_hash_iterator_key (place)));
-             m4_symbol_del (symbol);
-           }
-       }
-    }
-}
+static int symtab_print_list (const char *name, m4_symbol *symbol, void 
*ignored);
+static void symtab_dump (void);
 
-/* The following function is used for the cases where we want to do
-   something to each and every symbol in the table.  The function
-   m4_symtab_apply () traverses the symbol table, and calls a specified
-   function FUNC for each symbol in the table.  FUNC is called with a
-   pointer to the symbol, and the DATA argument.  */
-int
-m4_symtab_apply (m4_symtab_apply_func *func, void *data)
+static void
+symtab_dump (void)
 {
-  int result = 0;
   m4_hash_iterator *place = 0;
 
-  assert (func);
-
   while ((place = m4_hash_iterator_next (m4_symtab, place)))
     {
-      const char *name = (const char *) m4_hash_iterator_key (place);
-      m4_symbol *symbol = (m4_symbol *) m4_hash_iterator_value (place);
-      result = (*func) (name, symbol, data);
+      const char   *symbol_name        = (const char *) m4_hash_iterator_key 
(place);
+      m4_symbol           *symbol      = m4_hash_iterator_value (place);
+      m4_token    *token       = SYMBOL_TOKEN (symbol);
+      int          flags       = token ? SYMBOL_FLAGS (symbol) : 0;
+      lt_dlhandle   handle     = token ? SYMBOL_HANDLE (symbol) : 0;
+      const char   *module_name        = handle ? m4_module_name (handle) : 
"NONE";
+      const m4_builtin *bp;
+
+      fprintf (stderr, "%10s: (%d%s) %s=",
+              module_name, flags,
+              SYMBOL_TRACED (symbol) ? "!" : "", symbol_name);
 
-      if (result != 0)
-       break;
+      if (!token)
+       fputs ("<!UNDEFINED!>", stderr);
+      else
+       switch (SYMBOL_TYPE (symbol))
+         {
+         case M4_TOKEN_TEXT:
+           fputs (SYMBOL_TEXT (symbol), stderr);
+           break;
+
+         case M4_TOKEN_FUNC:
+           bp = m4_builtin_find_by_func (m4_module_builtins (handle),
+                                       SYMBOL_FUNC (symbol));
+           fprintf (stderr, "<%s>",
+                    bp ? bp->name : "!ERROR!");
+           break;
+         case M4_TOKEN_VOID:
+           fputs ("<!VOID!>", stderr);
+           break;
+       }
+      fputc ('\n', stderr);
     }
-
-  return result;
 }
 
-
-
-#ifdef DEBUG_SYM
-
-static int symtab_print_list (const char *name, m4_symbol *symbol, void 
*ignored);
-
 static void
 symtab_debug (void)
 {
-  m4_token_data_t t;
-  m4_token_data td;
+  m4_token_t t;
+  m4_token token;
   const char *text;
   m4_symbol *s;
   int delete;
 
-  while ((t = m4_next_token (&td)) != M4_TOKEN_EOF)
+  while ((t = m4_next_token (&token)) != M4_TOKEN_EOF)
     {
       if (t != M4_TOKEN_WORD)
        continue;
-      text = M4_TOKEN_DATA_TEXT (&td);
+      text = TOKEN_TEXT (&token);
       if (*text == '_')
        {
          delete = 1;
@@ -382,9 +414,10 @@ symtab_debug (void)
 static int
 symtab_print_list (const char *name, m4_symbol *symbol, void *ignored)
 {
-  printf ("\tname %s, addr %#x, flags: %s\n",
+  printf ("\tname %s, addr %#x, flags: %d %s\n",
          name, (unsigned) symbol,
-         M4_SYMBOL_TRACED (symbol) ? " traced" : "<none>");
+         SYMBOL_FLAGS (symbol),
+         SYMBOL_TRACED (symbol) ? " traced" : "<none>");
   return 0;
 }
 
Index: m4/utility.c
===================================================================
RCS file: /cvsroot/m4/m4/m4/utility.c,v
retrieving revision 1.17
diff -u -p -u -r1.17 utility.c
--- m4/utility.c 2001/10/04 07:47:29 1.17
+++ m4/utility.c 2001/10/11 21:01:09
@@ -78,22 +78,22 @@ m4_string ecomm;
    Since they are functions the caller does not need access to the
    internal data structure, so they are safe to export for use in
    external modules.  */
-m4_token_data_t
-m4_token_data_type (m4_token_data *name)
+m4_token_t
+m4_token_type (m4_token *name)
 {
-  return M4_TOKEN_DATA_TYPE(name);
+  return TOKEN_TYPE (name);
 }
 
 char *
-m4_token_data_text (m4_token_data *name)
+m4_token_text (m4_token *name)
 {
-  return M4_TOKEN_DATA_TEXT(name);
+  return TOKEN_TEXT (name);
 }
 
 m4_builtin_func *
-m4_token_data_func (m4_token_data *name)
+m4_token_func (m4_token *name)
 {
-  return M4_TOKEN_DATA_FUNC(name);
+  return TOKEN_FUNC (name);
 }
 
 
@@ -104,13 +104,13 @@ m4_token_data_func (m4_token_data *name)
    negative if not applicable, MAX is the maximum number, negative if not
    applicable.  */
 boolean
-m4_bad_argc (m4_token_data *name, int argc, int min, int max)
+m4_bad_argc (m4_token *token, int argc, int min, int max)
 {
   if (min > 0 && argc < min)
     {
       M4WARN ((warning_status, 0,
               _("Warning: %s: too few arguments: %d < %d"),
-              M4_TOKEN_DATA_TEXT (name), argc, min));
+              TOKEN_TEXT (token), argc, min));
       return TRUE;
     }
 
@@ -118,7 +118,7 @@ m4_bad_argc (m4_token_data *name, int ar
     {
       M4WARN ((warning_status, 0,
               _("Warning: %s: too many arguments (ignored): %d > %d"),
-              M4_TOKEN_DATA_TEXT (name), argc, max));
+              TOKEN_TEXT (token), argc, max));
       /* Return FALSE, otherwise it is not exactly `ignored'. */
       return FALSE;
     }
@@ -138,7 +138,7 @@ m4_skip_space (const char *arg)
    VALUEP. If the conversion fails, print error message for macro MACRO.
    Return TRUE iff conversion succeeds.  */
 boolean
-m4_numeric_arg (m4_token_data *macro, const char *arg, int *valuep)
+m4_numeric_arg (m4_token *macro, const char *arg, int *valuep)
 {
   char *endp;
 
@@ -147,7 +147,7 @@ m4_numeric_arg (m4_token_data *macro, co
     {
       M4WARN ((warning_status, 0,
               _("Warning: %s: non-numeric argument: %s"),
-              M4_TOKEN_DATA_TEXT (macro), arg));
+              TOKEN_TEXT (macro), arg));
       return FALSE;
     }
   return TRUE;
@@ -157,7 +157,7 @@ m4_numeric_arg (m4_token_data *macro, co
 /* Print ARGC arguments from the table ARGV to obstack OBS, separated by
    SEP, and quoted by the current quotes, if QUOTED is TRUE.  */
 void
-m4_dump_args (struct obstack *obs, int argc, m4_token_data **argv,
+m4_dump_args (struct obstack *obs, int argc, m4_token **argv,
              const char *sep, boolean quoted)
 {
   int i;
@@ -226,7 +226,7 @@ dumpdef_cmp (const void *s1, const void 
 int
 m4_dump_symbol (const char *name, m4_symbol *symbol, void *data)
 {
-  if (M4_SYMBOL_TYPE (symbol) != M4_TOKEN_VOID)
+  if (SYMBOL_TYPE (symbol) != M4_TOKEN_VOID)
     {
       struct m4_dump_symbol_data *symbol_data
        = (struct m4_dump_symbol_data *) data;
@@ -243,7 +243,7 @@ m4_dump_symbol (const char *name, m4_sym
    symbols, otherwise, only the specified symbols.  */
 void
 m4_dump_symbols (struct m4_dump_symbol_data *data, int argc,
-                m4_token_data **argv, boolean complain)
+                m4_token **argv, boolean complain)
 {
   data->base = (const char **) obstack_base (data->obs);
   data->size = 0;
@@ -260,7 +260,7 @@ m4_dump_symbols (struct m4_dump_symbol_d
       for (i = 1; i < argc; i++)
        {
          symbol = m4_symbol_lookup (M4ARG (i));
-         if (symbol != NULL && M4_SYMBOL_TYPE (symbol) != M4_TOKEN_VOID)
+         if (symbol != NULL && SYMBOL_TYPE (symbol) != M4_TOKEN_VOID)
            m4_dump_symbol (M4ARG (i), symbol, data);
          else if (complain)
            M4WARN ((warning_status, 0,
Index: modules/evalparse.c
===================================================================
RCS file: /cvsroot/m4/m4/modules/evalparse.c,v
retrieving revision 1.2
diff -u -p -u -r1.2 evalparse.c
--- modules/evalparse.c 2001/09/30 14:43:38 1.2
+++ modules/evalparse.c 2001/10/11 21:01:11
@@ -766,7 +766,7 @@ simple_term (eval_token et, number *v1)
 
 /* Main entry point, called from "eval" and "mpeval" builtins.  */
 void
-m4_evaluate (struct obstack *obs, int argc, m4_token_data **argv)
+m4_evaluate (struct obstack *obs, int argc, m4_token **argv)
 {
   int          radix   = 10;
   int          min     = 1;
Index: modules/format.c
===================================================================
RCS file: /cvsroot/m4/m4/modules/format.c,v
retrieving revision 1.12
diff -u -p -u -r1.12 format.c
--- modules/format.c 2001/10/01 07:38:48 1.12
+++ modules/format.c 2001/10/11 21:01:12
@@ -50,10 +50,10 @@
 /* The main formatting function.  Output is placed on the obstack OBS, the
    first argument in ARGV is the formatting string, and the rest is
    arguments for the string.  */
-void format (struct obstack *obs, int argc, m4_token_data **argv);
+void format (struct obstack *obs, int argc, m4_token **argv);
 
 void
-format (struct obstack *obs, int argc, m4_token_data **argv)
+format (struct obstack *obs, int argc, m4_token **argv)
 {
   char *fmt;                   /* format control string */
   const char *fstart;          /* beginning of current format spec */
Index: modules/gnu.c
===================================================================
RCS file: /cvsroot/m4/m4/modules/gnu.c,v
retrieving revision 1.15
diff -u -p -u -r1.15 gnu.c
--- modules/gnu.c 2001/10/07 16:33:08 1.15
+++ modules/gnu.c 2001/10/11 21:01:13
@@ -119,7 +119,10 @@ m4_macro m4_macro_table[] =
   { 0, 0 },
 };
 
-static void substitute (struct obstack *obs, const char *victim, const char 
*repl, struct re_registers *regs);
+static void substitute (struct obstack *obs, const char *victim,
+                       const char *repl, struct re_registers *regs);
+static void m4_patsubst_do (struct obstack *obs, int argc, m4_token **argv,
+                           int syntax);
 
 
 /* The builtin "builtin" allows calls to builtin macros, even if their
@@ -310,7 +313,7 @@ m4_regexp_compile (const char *caller,
  **/
 
 static void
-m4_regexp_do (struct obstack *obs, int argc, m4_token_data **argv,
+m4_regexp_do (struct obstack *obs, int argc, m4_token **argv,
              int syntax)
 {
   const char *victim;          /* first argument */
@@ -377,7 +380,8 @@ M4BUILTIN_HANDLER (eregexp)
 /**
  * patsubst(STRING, REGEXP, [REPLACEMENT])
  **/
-m4_patsubst_do (struct obstack *obs, int argc, m4_token_data **argv,
+static void
+m4_patsubst_do (struct obstack *obs, int argc, m4_token **argv,
                int syntax)
 {
   const char *victim;          /* first argument */
@@ -498,7 +502,7 @@ M4BUILTIN_HANDLER (syncoutput)
   if (m4_bad_argc (argv[0], argc, 2, 2))
     return;
 
-  if (M4_TOKEN_DATA_TYPE (argv[1]) != M4_TOKEN_TEXT)
+  if (TOKEN_TYPE (argv[1]) != M4_TOKEN_TEXT)
     return;
 
   if (   M4ARG (1)[0] == '0'
Index: modules/m4.c
===================================================================
RCS file: /cvsroot/m4/m4/modules/m4.c,v
retrieving revision 1.25
diff -u -p -u -r1.25 m4.c
--- modules/m4.c 2001/10/01 07:38:48 1.25
+++ modules/m4.c 2001/10/11 21:01:15
@@ -92,7 +92,7 @@ typedef unsigned long int unumber;
 #endif
 
 
-static void    include         (int argc, m4_token_data **argv,
+static void    include         (int argc, m4_token **argv,
                                 boolean silent);
 static int     set_trace       (const char *name, m4_symbol *symbol,
                                 void *data);
@@ -151,42 +151,25 @@ M4BUILTIN_HANDLER (define)
   if (m4_bad_argc (argv[0], argc, 2, 3))
     return;
 
-  if (M4_TOKEN_DATA_TYPE (argv[1]) != M4_TOKEN_TEXT)
+  if (TOKEN_TYPE (argv[1]) != M4_TOKEN_TEXT)
     return;
 
   if (argc == 2)
     {
-      m4_macro_define (M4ARG (1), NULL, "");
+      m4_macro_define (M4ARG (1), NULL, "", 0);
       return;
     }
 
-  switch (M4_TOKEN_DATA_TYPE (argv[2]))
+  switch (TOKEN_TYPE (argv[2]))
     {
     case M4_TOKEN_TEXT:
-      m4_macro_define (M4ARG (1), NULL, M4ARG (2));
+      m4_macro_define (M4ARG (1), TOKEN_HANDLE (argv[2]),
+                      TOKEN_TEXT (argv[2]), TOKEN_FLAGS (argv[2]));
       return;
 
     case M4_TOKEN_FUNC:
-      {
-       lt_dlhandle  handle  = M4_TOKEN_DATA_HANDLE (argv[2]);
-       const m4_builtin  *builtin = 0;
-       m4_symbol *symbol = 0;
-
-       /* If we find a TOKEN_FUNC with no defining module, then
-          somewhere along the way we have lost the module handle.  */
-       assert (handle);
-
-       /* FIXME: Why go to all this effort just to find the macro_args
-          and blind_no_args flags?  May as well carry these around with
-          the traced flag, and save a _lot_ of lookup time.  */
-       builtin = m4_builtin_find_by_func (m4_module_builtins (handle),
-                                          M4_TOKEN_DATA_FUNC (argv[2]));
-
-       symbol = m4_builtin_define (M4ARG (1), handle, builtin);
-
-       /* Be sure to propogate the flags from the TOKEN_FUNC.  */
-       M4_SYMBOL_TRACED (symbol) = M4_TOKEN_TRACED (argv[2]);
-      }
+      m4_builtin_define (M4ARG (1), TOKEN_HANDLE (argv[2]),
+                        TOKEN_FUNC (argv[2]), TOKEN_FLAGS (argv[2]));
       return;
     }
 
@@ -211,35 +194,25 @@ M4BUILTIN_HANDLER (pushdef)
   if (m4_bad_argc (argv[0], argc, 2, 3))
     return;
 
-  if (M4_TOKEN_DATA_TYPE (argv[1]) != M4_TOKEN_TEXT)
+  if (TOKEN_TYPE (argv[1]) != M4_TOKEN_TEXT)
     return;
 
   if (argc == 2)
     {
-      m4_macro_pushdef (M4ARG (1), NULL, "");
+      m4_macro_pushdef (M4ARG (1), NULL, "", 0);
       return;
     }
 
-  switch (M4_TOKEN_DATA_TYPE (argv[2]))
+  switch (TOKEN_TYPE (argv[2]))
     {
     case M4_TOKEN_TEXT:
-      m4_macro_pushdef (M4ARG (1), NULL, M4ARG (2));
+      m4_macro_pushdef (M4ARG (1), TOKEN_HANDLE (argv[2]),
+                       TOKEN_TEXT (argv[2]), TOKEN_FLAGS (argv[2]));
       return;
 
     case M4_TOKEN_FUNC:
-      {
-       lt_dlhandle  handle  = M4_TOKEN_DATA_HANDLE (argv[2]);
-       const m4_builtin  *builtin = 0;
-
-       /* If we find a TOKEN_FUNC with no defining module, then
-          somewhere along the way we have lost the module handle.  */
-       assert (handle);
-
-       builtin = m4_builtin_find_by_func (m4_module_builtins (handle),
-                                          M4_TOKEN_DATA_FUNC (argv[2]));
-
-       m4_builtin_pushdef (M4ARG (1), handle, builtin);
-      }
+      m4_builtin_pushdef (M4ARG (1), TOKEN_HANDLE (argv[2]),
+                         TOKEN_FUNC (argv[2]), TOKEN_FLAGS (argv[2]));
       return;
     }
 
@@ -341,21 +314,21 @@ M4BUILTIN_HANDLER (dumpdef)
       m4_symbol *symbol = m4_symbol_lookup (data.base[0]);
 
       fprintf (stderr, "%s:\t", data.base[0]);
-      assert (M4_SYMBOL_TYPE (symbol) == M4_TOKEN_TEXT
-             || M4_SYMBOL_TYPE (symbol) == M4_TOKEN_FUNC
-             || M4_SYMBOL_TYPE (symbol) == M4_TOKEN_VOID);
-      switch (M4_SYMBOL_TYPE (symbol))
+      assert (SYMBOL_TYPE (symbol) == M4_TOKEN_TEXT
+             || SYMBOL_TYPE (symbol) == M4_TOKEN_FUNC
+             || SYMBOL_TYPE (symbol) == M4_TOKEN_VOID);
+      switch (SYMBOL_TYPE (symbol))
        {
        case M4_TOKEN_TEXT:
          if (debug_level & M4_DEBUG_TRACE_QUOTE)
            fprintf (stderr, "%s%s%s\n",
-                    lquote.string, M4_SYMBOL_TEXT (symbol), rquote.string);
+                    lquote.string, SYMBOL_TEXT (symbol), rquote.string);
          else
-           fprintf (stderr, "%s\n", M4_SYMBOL_TEXT (symbol));
+           fprintf (stderr, "%s\n", SYMBOL_TEXT (symbol));
          break;
 
        case M4_TOKEN_FUNC:
-         bp = m4_builtin_find_by_func (NULL, M4_SYMBOL_FUNC (symbol));
+         bp = m4_builtin_find_by_func (NULL, SYMBOL_FUNC (symbol));
          assert (bp);
          fprintf (stderr, "<%s>\n", bp->name);
          break;
@@ -385,15 +358,15 @@ M4BUILTIN_HANDLER (defn)
       return;
     }
 
-  switch (M4_SYMBOL_TYPE (symbol))
+  switch (SYMBOL_TYPE (symbol))
     {
     case M4_TOKEN_TEXT:
-      m4_shipout_string(obs, M4_SYMBOL_TEXT (symbol), 0, TRUE);
+      m4_shipout_string(obs, SYMBOL_TEXT (symbol), 0, TRUE);
       return;
 
     case M4_TOKEN_FUNC:
-      m4_push_macro (M4_SYMBOL_FUNC (symbol), M4_SYMBOL_HANDLE (symbol),
-                    M4_SYMBOL_TRACED (symbol));
+      m4_push_macro (SYMBOL_FUNC (symbol), SYMBOL_HANDLE (symbol),
+                    SYMBOL_FLAGS (symbol));
       return;
 
     case M4_TOKEN_VOID:
@@ -561,7 +534,7 @@ M4BUILTIN_HANDLER (changecom)
 /* Generic include function.  Include the file given by the first argument,
    if it exists.  Complain about inaccesible files iff SILENT is FALSE.  */
 static void
-include (int argc, m4_token_data **argv, boolean silent)
+include (int argc, m4_token **argv, boolean silent)
 {
   FILE *fp;
   char *name = NULL;
@@ -660,7 +633,7 @@ M4BUILTIN_HANDLER (m4wrap)
 static int
 set_trace (const char *name, m4_symbol *symbol, void *data)
 {
-  M4_SYMBOL_TRACED (symbol) = (boolean) (data != NULL);
+  SYMBOL_TRACED (symbol) = (boolean) (data != NULL);
   return 0;
 }
 
Index: src/freeze.c
===================================================================
RCS file: /cvsroot/m4/m4/src/freeze.c,v
retrieving revision 1.19
diff -u -p -u -r1.19 freeze.c
--- src/freeze.c 2001/09/30 22:26:57 1.19
+++ src/freeze.c 2001/10/11 21:01:15
@@ -130,22 +130,22 @@ produce_symbol_dump (FILE *file, m4_hash
     {
       const char   *symbol_name        = (const char *) m4_hash_iterator_key 
(place);
       m4_symbol           *symbol      = m4_hash_iterator_value (place);
-      lt_dlhandle   handle     = M4_SYMBOL_HANDLE (symbol);
+      lt_dlhandle   handle     = SYMBOL_HANDLE (symbol);
       const char   *module_name        = handle ? m4_module_name (handle) : 
NULL;
       const m4_builtin *bp;
 
-      switch (M4_SYMBOL_TYPE (symbol))
+      switch (SYMBOL_TYPE (symbol))
        {
        case M4_TOKEN_TEXT:
          fprintf (file, "T%lu,%lu",
                   (unsigned long) strlen (symbol_name),
-                  (unsigned long) strlen (M4_SYMBOL_TEXT (symbol)));
+                  (unsigned long) strlen (SYMBOL_TEXT (symbol)));
          if (handle)
            fprintf (file, ",%lu", (unsigned long) strlen (module_name));
          fputc ('\n', file);
 
          fputs (symbol_name, file);
-         fputs (M4_SYMBOL_TEXT (symbol), file);
+         fputs (SYMBOL_TEXT (symbol), file);
          if (handle)
            fputs (module_name, file);
          fputc ('\n', file);
@@ -153,8 +153,8 @@ produce_symbol_dump (FILE *file, m4_hash
 
        case M4_TOKEN_FUNC:
          bp = m4_builtin_find_by_func
-               (m4_module_builtins (M4_SYMBOL_HANDLE(symbol)),
-                M4_SYMBOL_FUNC (symbol));
+               (m4_module_builtins (SYMBOL_HANDLE (symbol)),
+                SYMBOL_FUNC (symbol));
 
          if (bp == NULL)
            {
@@ -472,7 +472,16 @@ reload_frozen_state (const char *name)
            bp = m4_builtin_find_by_name (bt, string[1]);
 
          if (bp)
-           m4_builtin_pushdef (string[0], handle, bp);
+           {
+             int flags = 0;
+
+             if (bp->groks_macro_args)
+               BIT_SET (flags, TOKEN_MACRO_ARGS_BIT);
+             if (bp->blind_if_no_args)
+               BIT_SET (flags, TOKEN_BLIND_ARGS_BIT);
+
+             m4_builtin_pushdef (string[0], handle, bp->func, flags);
+           }
          else
            M4ERROR ((warning_status, 0,
                      _("`%s' from frozen file not found in builtin table!"),
@@ -647,7 +656,7 @@ reload_frozen_state (const char *name)
              if (strcmp (m4_module_name (handle), string[2]) == 0)
                break;
 
-         m4_macro_pushdef (string[0], handle, string[1]);
+         m4_macro_pushdef (string[0], handle, string[1], 0);
        }
        break;
 
Index: src/main.c
===================================================================
RCS file: /cvsroot/m4/m4/src/main.c,v
retrieving revision 1.28
diff -u -p -u -r1.28 main.c
--- src/main.c 2001/10/04 21:14:37 1.28
+++ src/main.c 2001/10/11 21:01:17
@@ -435,7 +435,7 @@ warranty; not even for MERCHANTABILITY o
            macro_value = "";
          else
            *macro_value++ = '\0';
-         m4_macro_define (defines->macro, NULL, macro_value);
+         m4_macro_define (defines->macro, NULL, macro_value, 0x0);
          break;
 
        case 'U':
@@ -444,7 +444,7 @@ warranty; not even for MERCHANTABILITY o
 
        case 't':
          symbol = m4_symbol_define (defines->macro);
-         M4_SYMBOL_TRACED (symbol) = TRUE;
+         SYMBOL_TRACED (symbol) = TRUE;
          break;
 
        case 'm':
Index: tests/builtins.at
===================================================================
RCS file: /cvsroot/m4/m4/tests/builtins.at,v
retrieving revision 1.5
diff -u -p -u -r1.5 builtins.at
--- tests/builtins.at 2001/09/20 08:49:31 1.5
+++ tests/builtins.at 2001/10/11 21:01:19
@@ -794,10 +794,7 @@ AT_DATA([[expout]],
 
 AT_DATA([[experr]],
 [[m4trace: -1- define(`my_define', <define>)
-m4trace: -1- my_define(`foo', `bar')
-m4trace: -1- my_define(`define', <define>)
 m4trace: -1- define(`my_define', <define>)
-m4trace: -1- my_define(`foo', `bar')
 m4trace: -1- define(`foo', `bar')
 ]])
 
@@ -859,10 +856,7 @@ AT_DATA([[expout]],
 
 AT_DATA([[experr]],
 [[m4trace: -1- define(`my_define', <define>)
-m4trace: -1- my_define(`foo', `bar')
-m4trace: -1- my_define(`define', <define>)
 m4trace: -1- define(`my_define', <define>)
-m4trace: -1- my_define(`foo', `bar')
 m4trace: -1- define(`foo', `bar')
 ]])
 
Index: tests/modules.at
===================================================================
RCS file: /cvsroot/m4/m4/tests/modules.at,v
retrieving revision 1.5
diff -u -p -u -r1.5 modules.at
--- tests/modules.at 2001/10/05 07:50:26 1.5
+++ tests/modules.at 2001/10/11 21:01:19
@@ -331,3 +331,43 @@ Test module unloaded.
 AT_CHECK_M4([-m load input.m4], 0, expout, experr)
 
 AT_CLEANUP
+
+
+## -------------------- ##
+## trace module symbols ##
+## -------------------- ##
+
+# The trace bit should not be lost if a builtin is unloaded from
+# memory and then redefined by a subsequent load.
+
+AT_SETUP([modules: trace])
+
+AT_DATA([[input.m4]],
+[[test
+load(`shadow')
+test
+unload(`shadow')
+test
+load(`shadow')
+test
+]])
+
+AT_DATA([[expout]],
+[[test
+Shadow module loaded.
+Shadow::test called.
+
+test
+Shadow module loaded.
+Shadow::test called.
+]])
+
+AT_DATA([[experr]],
+[[m4trace: -1- test -> `Shadow::`test' called.'
+m4trace: -1- test -> `Shadow::`test' called.'
+]])
+
+
+AT_CHECK_M4([-m load -t test input.m4], 0, expout, experr)
+
+AT_CLEANUP

-- 
  ())_. Gary V. Vaughan     gary@(oranda.demon.co.uk|gnu.org)
  ( '/  Research Scientist  http://www.oranda.demon.co.uk       ,_())____
  / )=  GNU Hacker          http://www.gnu.org/software/libtool  \'      `&
`(_~)_  Tech' Author        http://sources.redhat.com/autobook   =`---d__/



reply via email to

[Prev in Thread] Current Thread [Next in Thread]