pspp-cvs
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Pspp-cvs] pspp ./TODO src/ChangeLog src/procedure.c src/p...


From: Ben Pfaff
Subject: [Pspp-cvs] pspp ./TODO src/ChangeLog src/procedure.c src/p...
Date: Thu, 04 May 2006 06:19:03 +0000

CVSROOT:        /cvsroot/pspp
Module name:    pspp
Branch:         
Changes by:     Ben Pfaff <address@hidden>      06/05/04 06:19:02

Modified files:
        .              : TODO 
        src            : ChangeLog procedure.c procedure.h 
        src/data       : ChangeLog automake.mk dictionary.c variable.h 
        src/language/control: ChangeLog do-if.c loop.c repeat.c 
                              temporary.c 
        src/language/data-io: ChangeLog data-list.c get.c inpt-pgm.c 
                              matrix-data.c print.c 
        src/language/dictionary: apply-dictionary.c formats.c 
                                 missing-values.c modify-variables.c 
                                 numeric.c rename-variables.c 
                                 split-file.c sys-file-info.c 
                                 value-labels.c variable-display.c 
                                 variable-label.c vector.c weight.c 
        src/language/expressions: parse.c 
        src/language/lexer: variable-parser.c 
        src/language/stats: ChangeLog aggregate.c autorecode.c 
                            correlations.q descriptives.c flip.c means.q 
                            rank.q regression.q sort-cases.c 
        src/language/utilities: set.q 
        src/language/xforms: compute.c count.c recode.c sample.c 
                             select-if.c 
        src/math       : ChangeLog sort.c 
        src/ui/terminal: ChangeLog main.c 
Added files:
        src/data       : transformations.c transformations.h 

Log message:
        Continue reforming procedure execution.  In this phase, get rid of
        many global variables, consolidating procedure execution in
        procedure.c.  Encapsulate transformations in new "struct trns_chain".
        Also, change implementation of N OF CASES, FILTER, and PROCESS IF from
        special cases to transformations.

CVSWeb URLs:
http://cvs.savannah.gnu.org/viewcvs/pspp/pspp/TODO.diff?tr1=1.35&tr2=1.36&r1=text&r2=text
http://cvs.savannah.gnu.org/viewcvs/pspp/pspp/src/ChangeLog.diff?tr1=1.283&tr2=1.284&r1=text&r2=text
http://cvs.savannah.gnu.org/viewcvs/pspp/pspp/src/procedure.c.diff?tr1=1.12&tr2=1.13&r1=text&r2=text
http://cvs.savannah.gnu.org/viewcvs/pspp/pspp/src/procedure.h.diff?tr1=1.4&tr2=1.5&r1=text&r2=text
http://cvs.savannah.gnu.org/viewcvs/pspp/pspp/src/data/ChangeLog.diff?tr1=1.28&tr2=1.29&r1=text&r2=text
http://cvs.savannah.gnu.org/viewcvs/pspp/pspp/src/data/automake.mk.diff?tr1=1.4&tr2=1.5&r1=text&r2=text
http://cvs.savannah.gnu.org/viewcvs/pspp/pspp/src/data/dictionary.c.diff?tr1=1.9&tr2=1.10&r1=text&r2=text
http://cvs.savannah.gnu.org/viewcvs/pspp/pspp/src/data/transformations.c?rev=1.1
http://cvs.savannah.gnu.org/viewcvs/pspp/pspp/src/data/transformations.h?rev=1.1
http://cvs.savannah.gnu.org/viewcvs/pspp/pspp/src/data/variable.h.diff?tr1=1.7&tr2=1.8&r1=text&r2=text
http://cvs.savannah.gnu.org/viewcvs/pspp/pspp/src/language/control/ChangeLog.diff?tr1=1.1&tr2=1.2&r1=text&r2=text
http://cvs.savannah.gnu.org/viewcvs/pspp/pspp/src/language/control/do-if.c.diff?tr1=1.3&tr2=1.4&r1=text&r2=text
http://cvs.savannah.gnu.org/viewcvs/pspp/pspp/src/language/control/loop.c.diff?tr1=1.4&tr2=1.5&r1=text&r2=text
http://cvs.savannah.gnu.org/viewcvs/pspp/pspp/src/language/control/repeat.c.diff?tr1=1.5&tr2=1.6&r1=text&r2=text
http://cvs.savannah.gnu.org/viewcvs/pspp/pspp/src/language/control/temporary.c.diff?tr1=1.2&tr2=1.3&r1=text&r2=text
http://cvs.savannah.gnu.org/viewcvs/pspp/pspp/src/language/data-io/ChangeLog.diff?tr1=1.13&tr2=1.14&r1=text&r2=text
http://cvs.savannah.gnu.org/viewcvs/pspp/pspp/src/language/data-io/data-list.c.diff?tr1=1.13&tr2=1.14&r1=text&r2=text
http://cvs.savannah.gnu.org/viewcvs/pspp/pspp/src/language/data-io/get.c.diff?tr1=1.6&tr2=1.7&r1=text&r2=text
http://cvs.savannah.gnu.org/viewcvs/pspp/pspp/src/language/data-io/inpt-pgm.c.diff?tr1=1.8&tr2=1.9&r1=text&r2=text
http://cvs.savannah.gnu.org/viewcvs/pspp/pspp/src/language/data-io/matrix-data.c.diff?tr1=1.5&tr2=1.6&r1=text&r2=text
http://cvs.savannah.gnu.org/viewcvs/pspp/pspp/src/language/data-io/print.c.diff?tr1=1.5&tr2=1.6&r1=text&r2=text
http://cvs.savannah.gnu.org/viewcvs/pspp/pspp/src/language/dictionary/apply-dictionary.c.diff?tr1=1.4&tr2=1.5&r1=text&r2=text
http://cvs.savannah.gnu.org/viewcvs/pspp/pspp/src/language/dictionary/formats.c.diff?tr1=1.4&tr2=1.5&r1=text&r2=text
http://cvs.savannah.gnu.org/viewcvs/pspp/pspp/src/language/dictionary/missing-values.c.diff?tr1=1.4&tr2=1.5&r1=text&r2=text
http://cvs.savannah.gnu.org/viewcvs/pspp/pspp/src/language/dictionary/modify-variables.c.diff?tr1=1.3&tr2=1.4&r1=text&r2=text
http://cvs.savannah.gnu.org/viewcvs/pspp/pspp/src/language/dictionary/numeric.c.diff?tr1=1.6&tr2=1.7&r1=text&r2=text
http://cvs.savannah.gnu.org/viewcvs/pspp/pspp/src/language/dictionary/rename-variables.c.diff?tr1=1.2&tr2=1.3&r1=text&r2=text
http://cvs.savannah.gnu.org/viewcvs/pspp/pspp/src/language/dictionary/split-file.c.diff?tr1=1.2&tr2=1.3&r1=text&r2=text
http://cvs.savannah.gnu.org/viewcvs/pspp/pspp/src/language/dictionary/sys-file-info.c.diff?tr1=1.5&tr2=1.6&r1=text&r2=text
http://cvs.savannah.gnu.org/viewcvs/pspp/pspp/src/language/dictionary/value-labels.c.diff?tr1=1.3&tr2=1.4&r1=text&r2=text
http://cvs.savannah.gnu.org/viewcvs/pspp/pspp/src/language/dictionary/variable-display.c.diff?tr1=1.4&tr2=1.5&r1=text&r2=text
http://cvs.savannah.gnu.org/viewcvs/pspp/pspp/src/language/dictionary/variable-label.c.diff?tr1=1.4&tr2=1.5&r1=text&r2=text
http://cvs.savannah.gnu.org/viewcvs/pspp/pspp/src/language/dictionary/vector.c.diff?tr1=1.3&tr2=1.4&r1=text&r2=text
http://cvs.savannah.gnu.org/viewcvs/pspp/pspp/src/language/dictionary/weight.c.diff?tr1=1.2&tr2=1.3&r1=text&r2=text
http://cvs.savannah.gnu.org/viewcvs/pspp/pspp/src/language/expressions/parse.c.diff?tr1=1.6&tr2=1.7&r1=text&r2=text
http://cvs.savannah.gnu.org/viewcvs/pspp/pspp/src/language/lexer/variable-parser.c.diff?tr1=1.2&tr2=1.3&r1=text&r2=text
http://cvs.savannah.gnu.org/viewcvs/pspp/pspp/src/language/stats/ChangeLog.diff?tr1=1.17&tr2=1.18&r1=text&r2=text
http://cvs.savannah.gnu.org/viewcvs/pspp/pspp/src/language/stats/aggregate.c.diff?tr1=1.5&tr2=1.6&r1=text&r2=text
http://cvs.savannah.gnu.org/viewcvs/pspp/pspp/src/language/stats/autorecode.c.diff?tr1=1.4&tr2=1.5&r1=text&r2=text
http://cvs.savannah.gnu.org/viewcvs/pspp/pspp/src/language/stats/correlations.q.diff?tr1=1.5&tr2=1.6&r1=text&r2=text
http://cvs.savannah.gnu.org/viewcvs/pspp/pspp/src/language/stats/descriptives.c.diff?tr1=1.5&tr2=1.6&r1=text&r2=text
http://cvs.savannah.gnu.org/viewcvs/pspp/pspp/src/language/stats/flip.c.diff?tr1=1.6&tr2=1.7&r1=text&r2=text
http://cvs.savannah.gnu.org/viewcvs/pspp/pspp/src/language/stats/means.q.diff?tr1=1.3&tr2=1.4&r1=text&r2=text
http://cvs.savannah.gnu.org/viewcvs/pspp/pspp/src/language/stats/rank.q.diff?tr1=1.6&tr2=1.7&r1=text&r2=text
http://cvs.savannah.gnu.org/viewcvs/pspp/pspp/src/language/stats/regression.q.diff?tr1=1.20&tr2=1.21&r1=text&r2=text
http://cvs.savannah.gnu.org/viewcvs/pspp/pspp/src/language/stats/sort-cases.c.diff?tr1=1.2&tr2=1.3&r1=text&r2=text
http://cvs.savannah.gnu.org/viewcvs/pspp/pspp/src/language/utilities/set.q.diff?tr1=1.6&tr2=1.7&r1=text&r2=text
http://cvs.savannah.gnu.org/viewcvs/pspp/pspp/src/language/xforms/compute.c.diff?tr1=1.3&tr2=1.4&r1=text&r2=text
http://cvs.savannah.gnu.org/viewcvs/pspp/pspp/src/language/xforms/count.c.diff?tr1=1.3&tr2=1.4&r1=text&r2=text
http://cvs.savannah.gnu.org/viewcvs/pspp/pspp/src/language/xforms/recode.c.diff?tr1=1.4&tr2=1.5&r1=text&r2=text
http://cvs.savannah.gnu.org/viewcvs/pspp/pspp/src/language/xforms/sample.c.diff?tr1=1.4&tr2=1.5&r1=text&r2=text
http://cvs.savannah.gnu.org/viewcvs/pspp/pspp/src/language/xforms/select-if.c.diff?tr1=1.3&tr2=1.4&r1=text&r2=text
http://cvs.savannah.gnu.org/viewcvs/pspp/pspp/src/math/ChangeLog.diff?tr1=1.3&tr2=1.4&r1=text&r2=text
http://cvs.savannah.gnu.org/viewcvs/pspp/pspp/src/math/sort.c.diff?tr1=1.6&tr2=1.7&r1=text&r2=text
http://cvs.savannah.gnu.org/viewcvs/pspp/pspp/src/ui/terminal/ChangeLog.diff?tr1=1.8&tr2=1.9&r1=text&r2=text
http://cvs.savannah.gnu.org/viewcvs/pspp/pspp/src/ui/terminal/main.c.diff?tr1=1.10&tr2=1.11&r1=text&r2=text

Patches:
Index: pspp/TODO
diff -u pspp/TODO:1.35 pspp/TODO:1.36
--- pspp/TODO:1.35      Wed Apr 26 22:15:43 2006
+++ pspp/TODO   Thu May  4 06:19:02 2006
@@ -1,4 +1,18 @@
-Time-stamp: <2006-04-26 15:15:36 blp>
+Time-stamp: <2006-05-03 22:28:57 blp>
+
+Procedure processing:
+
+* Should not need temporary casefile in the common case.
+
+* All of the procedure_*() variants can (and should) be implemented in terms of
+  a variant that provides "proc_func" plus an "end_func" called after all
+  processing is complete.
+
+* The "split" variants should not dump the splits to the output file
+  automatically.  There is no need for the procedure code to talk to the output
+  manager.
+
+* LAG need not be as much of a special case.
 
 Get rid of need for GNU diff in `make check'.
 
Index: pspp/src/ChangeLog
diff -u pspp/src/ChangeLog:1.283 pspp/src/ChangeLog:1.284
--- pspp/src/ChangeLog:1.283    Thu Apr 27 03:01:57 2006
+++ pspp/src/ChangeLog  Thu May  4 06:19:02 2006
@@ -1,3 +1,70 @@
+Wed May  3 22:24:34 2006  Ben Pfaff  <address@hidden>
+
+       Continue reforming procedure execution.  In this phase, get rid of
+       many global variables, consolidating procedure execution in
+       procedure.c.  Encapsulate transformations in new "struct
+       trns_chain".  Also, change implementation of N OF CASES, FILTER,
+       and PROCESS IF from special cases to transformations.
+        
+       * procedure.c: (global var vfm_source) Make static.  Changed
+       external references to use in_input_program(), proc_set_source(),
+       or proc_capture_output() instead.
+       (global var vfm_sink) Make static.  Changed external references to
+       use proc_set_sink() instead.
+       (global var default_dict) Move here from data/dictionary.c.
+       (static var permanent_trns_chain) New var.
+       (static var temp_dict) Renamed permanent_dict, updated references.
+       (static var temporary_trns_chain) New var.
+       (static var cur_trns_chain) New var.
+       (static var in_procedure) Removed.
+       (global var t_trns) Removed.
+       (global var n_trns) Removed.
+       (global var m_trns) Removed.
+       (global var f_trns) Removed.
+       (procedure) Even if there's "nothing to do" we need to clear
+       PROCESS IF, N OF CASES, vector state.  (This should be
+       abstracted.)
+       (multipass_callback) New function.
+       (multipass_procedure) New function.
+       (open_active_file) Add N OF CASES, FILTER, PROCESS IF
+       transformations.  Finalize transformations.  No need to call
+       ctl_stack_clear() anymore because finalizers will do that.
+       (write_case) Simplify and rewrite.
+       (execute_transformations) Removed.
+       (filter_case) Removed.
+       (close_active_file) Use proc_cancel_temporary_transformations().
+       No need to clear PROCESS IF, N OF CASES here anymore because
+       helpers do that.
+       (multipass_procedure_with_splits) Keep track of success.
+       (multipass_split_callback) Ditto.
+       (multipass_split_output) Ditto.
+       (discard_variables) No need to call ctl_stack_clear() anymore
+       because finalizers will do that.
+       (proc_capture_transformations) New function.
+       (add_transformation) Rewrite in terms of trns_chain_append().
+       (add_transformation_with_finalizer) New function.
+       (next_transformation) Rewrite in terms of trns_chain_next().
+       (proc_in_temporary_transformations) New function.
+       (proc_start_temporary_transformations) New function.
+       (proc_make_temporary_transformations_permanent) New function.
+       (proc_cancel_temporary_transformations) New function.
+       (cancel_transformations) Rename proc_cancel_all_transformations(),
+       rewrite in terms of trns_chain_destroy().
+       (proc_init) New function.
+       (proc_done) New function.
+       (proc_set_sink) New function.
+       (proc_set_source) New function.
+       (proc_has_source) New function.
+       (proc_capture_output) New function.
+       (add_case_limit_trns) New function.
+       (case_limit_trns_proc) New function.
+       (case_limit_trns_free) New function.
+       (add_filter_trns) New function.
+       (filter_trns_proc) New function.
+       (add_process_if_trns) New function.
+       (process_if_trns_proc) New function.
+       (process_if_trns_free) New function.
+
 Wed Apr 26 20:00:00 2006  Ben Pfaff  <address@hidden>
 
        * procedure.c (create_trns_case): Fix inverted decision on whether
Index: pspp/src/data/ChangeLog
diff -u pspp/src/data/ChangeLog:1.28 pspp/src/data/ChangeLog:1.29
--- pspp/src/data/ChangeLog:1.28        Thu May  4 05:51:12 2006
+++ pspp/src/data/ChangeLog     Thu May  4 06:19:02 2006
@@ -1,3 +1,20 @@
+Wed May  3 22:42:12 2006  Ben Pfaff  <address@hidden>
+
+       Continue reforming procedure execution.  In this phase, get rid of
+       many global variables, consolidating procedure execution in
+       procedure.c.  Encapsulate transformations in new "struct
+       trns_chain".  Also, change implementation of N OF CASES, FILTER,
+       and PROCESS IF from special cases to transformations.
+        
+       * automake.mk: (src_data_libdata_a_SOURCES) Add transformations.c,
+       transformations.h.
+
+       * dictionary.c: (global variable default_dict) Move to
+       src/procedure.c.
+
+       * variable.h: (TRNS_*) Move to transformations.h.
+       (struct transformation) Move to transformations.c.
+
 Thu May  4 13:47:06 WST 2006 John Darrington <address@hidden>
 
        * sys-file-reader.c: Fixed invalid read problems.
Index: pspp/src/data/automake.mk
diff -u pspp/src/data/automake.mk:1.4 pspp/src/data/automake.mk:1.5
--- pspp/src/data/automake.mk:1.4       Thu Apr 27 02:42:01 2006
+++ pspp/src/data/automake.mk   Thu May  4 06:19:02 2006
@@ -57,6 +57,8 @@
        src/data/sys-file-reader.h \
        src/data/sys-file-writer.c \
        src/data/sys-file-writer.h \
+       src/data/transformations.c \
+       src/data/transformations.h \
        src/data/value.h \
        src/data/value-labels.c \
        src/data/value-labels.h \
Index: pspp/src/data/dictionary.c
diff -u pspp/src/data/dictionary.c:1.9 pspp/src/data/dictionary.c:1.10
--- pspp/src/data/dictionary.c:1.9      Tue May  2 03:51:43 2006
+++ pspp/src/data/dictionary.c  Thu May  4 06:19:02 2006
@@ -59,9 +59,6 @@
     size_t vector_cnt;          /* Number of vectors. */
   };
 
-/* Active file dictionary. */
-struct dictionary *default_dict;
-
 /* Creates and returns a new dictionary. */
 struct dictionary *
 dict_create (void) 
Index: pspp/src/data/variable.h
diff -u pspp/src/data/variable.h:1.7 pspp/src/data/variable.h:1.8
--- pspp/src/data/variable.h:1.7        Tue May  2 03:51:43 2006
+++ pspp/src/data/variable.h    Thu May  4 06:19:02 2006
@@ -28,8 +28,6 @@
 #include "format.h"
 #include "missing-values.h"
 
-/* Script variables. */
-
 /* Variable type. */
 enum var_type
   {
@@ -128,71 +126,15 @@
     int cnt;                   /* Number of variables. */
   };
 
-void discard_variables (void);
-
-/* This is the active file dictionary. */
-extern struct dictionary *default_dict;
-
-/* Transformation state. */
-
 /* PROCESS IF expression. */
 extern struct expression *process_if_expr;
 
-/* TEMPORARY support. */
-
-/* 1=TEMPORARY has been executed at some point. */
-extern int temporary;
-
-/* If temporary!=0, the saved dictionary. */
-extern struct dictionary *temp_dict;
-
-/* If temporary!=0, index into t_trns[] (declared far below) that
-   gives the point at which data should be written out.  -1 means that
-   the data shouldn't be changed since all transformations are
-   temporary. */
-extern size_t temp_trns;
-
-void cancel_temporary (void);
-
 struct ccase;
 void dump_split_vars (const struct ccase *);
 
-/* Transformations. */
-
-/* trns_proc_func return values. */
-#define TRNS_CONTINUE   -1 /* Continue to next transformation. */
-#define TRNS_DROP_CASE  -2 /* Drop this case. */
-#define TRNS_ERROR      -3 /* A serious error, so stop the procedure. */
-#define TRNS_NEXT_CASE  -4 /* Skip to next case.  INPUT PROGRAM only. */
-#define TRNS_END_FILE   -5 /* End of input.  INPUT PROGRAM only. */
-
-typedef int trns_proc_func (void *, struct ccase *, int);
-typedef bool trns_free_func (void *);
-
-/* A transformation. */
-struct transformation
-  {
-    trns_proc_func *proc;       /* Transformation proc. */
-    trns_free_func *free;       /* Garbage collector proc. */
-    void *private;              /* Private data. */
-  };
-
-/* Array of transformations */
-extern struct transformation *t_trns;
-
-/* Number of transformations, maximum number in array currently. */
-extern size_t n_trns, m_trns;
-
-/* Index of first transformation that is really a transformation.  Any
-   transformations before this belong to INPUT PROGRAM. */
-extern size_t f_trns;
-
-void add_transformation (trns_proc_func *, trns_free_func *, void *);
-size_t next_transformation (void);
-bool cancel_transformations (void);
-
 struct var_set;
 
+struct dictionary;
 struct var_set *var_set_create_from_dict (const struct dictionary *d);
 struct var_set *var_set_create_from_array (struct variable *const *var,
                                            size_t);
Index: pspp/src/language/control/ChangeLog
diff -u pspp/src/language/control/ChangeLog:1.1 
pspp/src/language/control/ChangeLog:1.2
--- pspp/src/language/control/ChangeLog:1.1     Sat Mar  4 01:11:57 2006
+++ pspp/src/language/control/ChangeLog Thu May  4 06:19:02 2006
@@ -1,3 +1,30 @@
+Wed May  3 22:45:41 2006  Ben Pfaff  <address@hidden>
+
+       Continue reforming procedure execution.  In this phase, get rid of
+       many global variables, consolidating procedure execution in
+       procedure.c.  Encapsulate transformations in new "struct
+       trns_chain".  Also, change implementation of N OF CASES, FILTER,
+       and PROCESS IF from special cases to transformations.
+        
+       * do-if.c: (cmd_do_if) Use finalizer to ensure control stack gets
+       cleared.
+       (do_if_finalize_func) New function.
+
+       * loop.c: (create_loop_trns) Use finalizer to ensure control stack gets
+       cleared.
+       (loop_trns_finalize) New function.
+
+       * temporary.c: (global var temporary) Removed.  Changed references
+       to use proc_make_temporary_transformations_permanent() or
+       proc_in_temporary_transformations().
+       (global var temp_dict) Removed.
+       (global var temp_trns) Removed.
+       (cmd_temporary) Reimplement in terms of
+       proc_in_temporary_transformations() and
+       proc_start_temporary_transformations().
+       (cancel_temporary) Moved to procedure.c, renamed
+       proc_cancel_temporary_transformations().
+               
 Thu Mar  2 08:40:33 WST 2006 John Darrington <address@hidden>
        
        * Moved files from src directory
Index: pspp/src/language/control/do-if.c
diff -u pspp/src/language/control/do-if.c:1.3 
pspp/src/language/control/do-if.c:1.4
--- pspp/src/language/control/do-if.c:1.3       Wed Mar 15 03:29:10 2006
+++ pspp/src/language/control/do-if.c   Thu May  4 06:19:02 2006
@@ -22,15 +22,17 @@
 #include <stdlib.h>
 
 #include "control-stack.h"
-#include <libpspp/message.h>
-#include <libpspp/alloc.h>
+#include <procedure.h>
+#include <data/transformations.h>
+#include <data/variable.h>
 #include <language/command.h>
-#include <libpspp/compiler.h>
-#include <libpspp/message.h>
 #include <language/expressions/public.h>
 #include <language/lexer/lexer.h>
+#include <libpspp/alloc.h>
+#include <libpspp/compiler.h>
+#include <libpspp/message.h>
+#include <libpspp/message.h>
 #include <libpspp/str.h>
-#include <data/variable.h>
 
 #include "gettext.h"
 #define _(msgid) gettext (msgid)
@@ -89,6 +91,7 @@
 static bool must_not_have_else (struct do_if_trns *);
 static void close_do_if (void *do_if);
 
+static trns_finalize_func do_if_finalize_func;
 static trns_proc_func do_if_trns_proc, break_trns_proc;
 static trns_free_func do_if_trns_free;
 
@@ -101,7 +104,8 @@
   do_if->clause_cnt = 0;
 
   ctl_stack_push (&do_if_class, do_if);
-  add_transformation (do_if_trns_proc, do_if_trns_free, do_if);
+  add_transformation_with_finalizer (do_if_finalize_func,
+                                     do_if_trns_proc, do_if_trns_free, do_if);
 
   return parse_clause (do_if);
 }
@@ -219,6 +223,17 @@
   clause->target_index = target_index;
 }
 
+/* Finalizes DO IF by clearing the control stack, thus ensuring
+   that all open DO IFs are closed. */ 
+static void
+do_if_finalize_func (void *do_if_ UNUSED) 
+{
+  /* This will be called multiple times if multiple DO IFs were
+     executed, which is slightly unclean, but at least it's
+     idempotent. */
+  ctl_stack_clear ();
+}
+
 /* DO IF transformation procedure.
    Checks each clause and jumps to the appropriate
    transformation. */
Index: pspp/src/language/control/loop.c
diff -u pspp/src/language/control/loop.c:1.4 
pspp/src/language/control/loop.c:1.5
--- pspp/src/language/control/loop.c:1.4        Wed Apr 26 20:54:08 2006
+++ pspp/src/language/control/loop.c    Thu May  4 06:19:02 2006
@@ -18,21 +18,23 @@
    02110-1301, USA. */
 
 #include <config.h>
-#include <libpspp/message.h>
-#include <libpspp/alloc.h>
+
+#include "control-stack.h"
 #include <data/case.h>
-#include <language/command.h>
-#include <libpspp/compiler.h>
 #include <data/dictionary.h>
-#include "control-stack.h"
-#include <libpspp/message.h>
+#include <procedure.h>
+#include <data/settings.h>
+#include <data/transformations.h>
+#include <data/variable.h>
+#include <language/command.h>
 #include <language/expressions/public.h>
 #include <language/lexer/lexer.h>
+#include <libpspp/alloc.h>
+#include <libpspp/compiler.h>
+#include <libpspp/message.h>
 #include <libpspp/misc.h>
 #include <libpspp/pool.h>
-#include <data/settings.h>
 #include <libpspp/str.h>
-#include <data/variable.h>
 
 #include "gettext.h"
 #define _(msgid) gettext (msgid)
@@ -79,6 +81,7 @@
 
 static struct ctl_class loop_class;
 
+static trns_finalize_func loop_trns_finalize;
 static trns_proc_func loop_trns_proc, end_loop_trns_proc, break_trns_proc;
 static trns_free_func loop_trns_free;
 
@@ -248,7 +251,8 @@
   loop->first_expr = loop->by_expr = loop->last_expr = NULL;
   loop->loop_condition = loop->end_loop_condition = NULL;
 
-  add_transformation (loop_trns_proc, loop_trns_free, loop);
+  add_transformation_with_finalizer (loop_trns_finalize,
+                                     loop_trns_proc, loop_trns_free, loop);
   loop->past_LOOP_index = next_transformation ();
 
   ctl_stack_push (&loop_class, loop);
@@ -256,6 +260,17 @@
   return loop;
 }
 
+/* Finalizes LOOP by clearing the control stack, thus ensuring
+   that all open LOOPs are closed. */ 
+static void
+loop_trns_finalize (void *do_if_ UNUSED) 
+{
+  /* This will be called multiple times if multiple LOOPs were
+     executed, which is slightly unclean, but at least it's
+     idempotent. */
+  ctl_stack_clear ();
+}
+
 /* Sets up LOOP for the first pass. */
 static int
 loop_trns_proc (void *loop_, struct ccase *c, int case_num)
Index: pspp/src/language/control/repeat.c
diff -u pspp/src/language/control/repeat.c:1.5 
pspp/src/language/control/repeat.c:1.6
--- pspp/src/language/control/repeat.c:1.5      Sun Apr 16 04:46:41 2006
+++ pspp/src/language/control/repeat.c  Thu May  4 06:19:02 2006
@@ -18,24 +18,29 @@
    02110-1301, USA. */
 
 #include <config.h>
+
 #include "repeat.h"
-#include <libpspp/message.h>
+
 #include <ctype.h>
 #include <math.h>
 #include <stdlib.h>
-#include <libpspp/alloc.h>
-#include <language/command.h>
+
 #include <data/dictionary.h>
-#include "intprops.h"
-#include <libpspp/message.h>
-#include <language/line-buffer.h>
+#include <procedure.h>
+#include <data/settings.h>
+#include <language/command.h>
 #include <language/lexer/lexer.h>
+#include <language/line-buffer.h>
+#include <libpspp/alloc.h>
+#include <libpspp/message.h>
+#include <libpspp/message.h>
 #include <libpspp/misc.h>
 #include <libpspp/pool.h>
-#include <data/settings.h>
 #include <libpspp/str.h>
 #include <data/variable.h>
 
+#include "intprops.h"
+
 #include "gettext.h"
 #define _(msgid) gettext (msgid)
 
Index: pspp/src/language/control/temporary.c
diff -u pspp/src/language/control/temporary.c:1.2 
pspp/src/language/control/temporary.c:1.3
--- pspp/src/language/control/temporary.c:1.2   Wed Mar 15 03:29:10 2006
+++ pspp/src/language/control/temporary.c       Thu May  4 06:19:02 2006
@@ -18,66 +18,35 @@
    02110-1301, USA. */
 
 #include <config.h>
-#include <libpspp/message.h>
+
 #include <stddef.h>
 #include <stdlib.h>
-#include <libpspp/alloc.h>
-#include <language/command.h>
-#include <data/dictionary.h>
+
 #include "control-stack.h"
-#include <libpspp/message.h>
-#include <libpspp/hash.h>
-#include <language/lexer/lexer.h>
-#include <libpspp/str.h>
+#include <data/dictionary.h>
+#include <procedure.h>
+#include <data/transformations.h>
 #include <data/value-labels.h>
 #include <data/variable.h>
+#include <language/command.h>
+#include <language/lexer/lexer.h>
+#include <libpspp/alloc.h>
+#include <libpspp/hash.h>
+#include <libpspp/message.h>
+#include <libpspp/message.h>
+#include <libpspp/str.h>
 
 #include "gettext.h"
 #define _(msgid) gettext (msgid)
 
-int temporary;
-struct dictionary *temp_dict;
-size_t temp_trns;
-
 /* Parses the TEMPORARY command. */
 int
 cmd_temporary (void)
 {
-  /* TEMPORARY is not allowed inside DO IF or LOOP. */
-  if (!ctl_stack_is_empty ())
-    {
-      msg (SE, _("This command is not valid inside DO IF or LOOP."));
-      return CMD_FAILURE;
-    }
-
-  /* TEMPORARY can only appear once! */
-  if (temporary)
-    {
-      msg (SE, _("This command may only appear once between "
-          "procedures and procedure-like commands."));
-      return CMD_FAILURE;
-    }
-
-  /* Make a copy of the current dictionary. */
-  temporary = 1;
-  temp_dict = dict_clone (default_dict);
-  temp_trns = n_trns;
-
+  if (!proc_in_temporary_transformations ())
+    proc_start_temporary_transformations ();
+  else
+    msg (SE, _("This command may only appear once between "
+               "procedures and procedure-like commands."));
   return lex_end_of_command ();
 }
-
-/* Cancels the temporary transformation, if any. */
-void
-cancel_temporary (void)
-{
-  if (temporary)
-    {
-      if (temp_dict) 
-        {
-          dict_destroy (temp_dict);
-          temp_dict = NULL; 
-        }
-      temporary = 0;
-      temp_trns = 0;
-    }
-}
Index: pspp/src/language/data-io/ChangeLog
diff -u pspp/src/language/data-io/ChangeLog:1.13 
pspp/src/language/data-io/ChangeLog:1.14
--- pspp/src/language/data-io/ChangeLog:1.13    Tue May  2 03:51:43 2006
+++ pspp/src/language/data-io/ChangeLog Thu May  4 06:19:02 2006
@@ -1,3 +1,23 @@
+Wed May  3 23:00:17 2006  Ben Pfaff  <address@hidden>
+
+       Continue reforming procedure execution.  In this phase, get rid of
+       many global variables, consolidating procedure execution in
+       procedure.c.  Encapsulate transformations in new "struct
+       trns_chain".  Also, change implementation of N OF CASES, FILTER,
+       and PROCESS IF from special cases to transformations.
+        
+       * data-list.c: (data_list_trns_proc) Return TRNS_END_FILE at end
+       of file.  (Why didn't we do this before?)
+       (cmd_match_files) Direct procedure output to null sink.
+       Use discard_variables() instead of indirect version.
+
+       * inpt-pgm.c: Use transformation chain.
+       (struct input_program_pgm) Add trns_chain member.
+       (cmd_input_program) Initialize trns_chain member and capture
+       transformations with proc_capture_transformations().
+       (input_program_source_read) Use trns_chain_execute().
+       (destroy_input_program) Destroy input chain.
+
 Tue May  2 10:39:56 WST 2006 John Darrington <address@hidden>
 
        * list.q Changed from using fixed length char buffers to struct 
Index: pspp/src/language/data-io/data-list.c
diff -u pspp/src/language/data-io/data-list.c:1.13 
pspp/src/language/data-io/data-list.c:1.14
--- pspp/src/language/data-io/data-list.c:1.13  Mon May  1 23:09:17 2006
+++ pspp/src/language/data-io/data-list.c       Thu May  4 06:19:02 2006
@@ -26,10 +26,12 @@
 
 #include <data/case-source.h>
 #include <data/case.h>
+#include <data/case-source.h>
 #include <data/data-in.h>
 #include <data/dictionary.h>
 #include <data/format.h>
 #include <data/settings.h>
+#include <data/transformations.h>
 #include <data/variable.h>
 #include <language/command.h>
 #include <language/data-io/data-reader.h>
@@ -101,6 +103,7 @@
 static void dump_free_table (const struct data_list_pgm *,
                              const struct file_handle *);
 static void destroy_dls_var_spec (struct dls_var_spec *);
+
 static trns_free_func data_list_trns_free;
 static trns_proc_func data_list_trns_proc;
 
@@ -255,10 +258,10 @@
   if (dls->reader == NULL)
     goto error;
 
-  if (vfm_source != NULL)
+  if (in_input_program ())
     add_transformation (data_list_trns_proc, data_list_trns_free, dls);
   else 
-    vfm_source = create_case_source (&data_list_source_class, dls);
+    proc_set_source (create_case_source (&data_list_source_class, dls));
 
   return CMD_SUCCESS;
 
@@ -534,7 +537,7 @@
       else
        {
          v = dict_lookup_var_assert (default_dict, fx->name[i]);
-         if (vfm_source == NULL)
+         if (!in_input_program ())
            {
              msg (SE, _("%s is a duplicate variable name."), fx->name[i]);
              return 0;
@@ -1228,7 +1231,7 @@
       retval = TRNS_ERROR;
     }
   else
-    retval = TRNS_DROP_CASE;
+    retval = TRNS_END_FILE;
   
   /* If there was an END subcommand handle it. */
   if (dls->end != NULL) 
@@ -1237,7 +1240,7 @@
       if (retval == TRNS_DROP_CASE)
         {
           *end = 1.0;
-          retval = TRNS_CONTINUE;
+          retval = TRNS_END_FILE;
         }
       else
         *end = 0.0;
Index: pspp/src/language/data-io/get.c
diff -u pspp/src/language/data-io/get.c:1.6 pspp/src/language/data-io/get.c:1.7
--- pspp/src/language/data-io/get.c:1.6 Thu Apr 27 02:42:01 2006
+++ pspp/src/language/data-io/get.c     Thu May  4 06:19:02 2006
@@ -31,6 +31,7 @@
 #include <data/settings.h>
 #include <data/storage-stream.h>
 #include <data/sys-file-writer.h>
+#include <data/transformations.h>
 #include <data/value-labels.h>
 #include <data/variable.h>
 #include <language/command.h>
@@ -147,7 +148,7 @@
   dict_destroy (default_dict);
   default_dict = dict;
 
-  vfm_source = create_case_source (&case_reader_source_class, pgm);
+  proc_set_source (create_case_source (&case_reader_source_class, pgm));
 
   return CMD_SUCCESS;
 
@@ -887,21 +888,18 @@
             }
           used_active_file = true;
 
-          if (vfm_source == NULL)
+          if (!proc_has_source ())
             {
               msg (SE, _("Cannot specify the active file since no active "
                          "file has been defined."));
               goto error;
             }
 
-          if (temporary != 0)
-            {
-              msg (SE,
-                   _("MATCH FILES may not be used after TEMPORARY when "
-                     "the active file is an input source.  "
-                     "Temporary transformations will be made permanent."));
-              cancel_temporary (); 
-            }
+          if (proc_make_temporary_transformations_permanent ())
+            msg (SE,
+                 _("MATCH FILES may not be used after TEMPORARY when "
+                   "the active file is an input source.  "
+                   "Temporary transformations will be made permanent."));
 
           file->dict = default_dict;
         }
@@ -1134,17 +1132,18 @@
     goto error;
 
   if (used_active_file) 
-    ok = procedure (mtf_processing, &mtf) && mtf_processing_finish (&mtf);
+    {
+      proc_set_sink (create_case_sink (&null_sink_class, default_dict, NULL));
+      ok = procedure (mtf_processing, &mtf) && mtf_processing_finish (&mtf); 
+    }
   else
     ok = mtf_processing_finish (&mtf);
 
-  free_case_source (vfm_source);
-  vfm_source = NULL;
+  discard_variables ();
 
-  dict_destroy (default_dict);
   default_dict = mtf.dict;
   mtf.dict = NULL;
-  vfm_source = mtf.sink->class->make_source (mtf.sink);
+  proc_set_source (mtf.sink->class->make_source (mtf.sink));
   free_case_sink (mtf.sink);
   
   if (!mtf_free (&mtf))
Index: pspp/src/language/data-io/inpt-pgm.c
diff -u pspp/src/language/data-io/inpt-pgm.c:1.8 
pspp/src/language/data-io/inpt-pgm.c:1.9
--- pspp/src/language/data-io/inpt-pgm.c:1.8    Tue May  2 01:29:06 2006
+++ pspp/src/language/data-io/inpt-pgm.c        Thu May  4 06:19:02 2006
@@ -28,6 +28,7 @@
 #include <data/case.h>
 #include <data/case-source.h>
 #include <data/dictionary.h>
+#include <data/transformations.h>
 #include <data/variable.h>
 #include <language/command.h>
 #include <language/data-io/data-reader.h>
@@ -64,6 +65,8 @@
 
 struct input_program_pgm 
   {
+    struct trns_chain *trns_chain;
+
     size_t case_nr;             /* Incremented by END CASE transformation. */
     write_case_func *write_case;/* Called by END CASE. */
     write_case_data wc_data;    /* Aux data used by END CASE. */
@@ -110,6 +113,7 @@
     return lex_end_of_command ();
 
   inp = xmalloc (sizeof *inp);
+  inp->trns_chain = NULL;
   inp->init = NULL;
   
   inside_input_program = true;
@@ -147,9 +151,8 @@
       return CMD_FAILURE;
     }
   
-  /* Mark the boundary between INPUT PROGRAM transformations and
-     ordinary transformations. */
-  f_trns = n_trns;
+  inp->trns_chain = proc_capture_transformations ();
+  trns_chain_finalize (inp->trns_chain);
 
   /* Figure out how to initialize each input case. */
   inp->init_cnt = dict_get_next_value_idx (default_dict);
@@ -172,8 +175,7 @@
     assert (inp->init[i] != -1);
   inp->case_size = dict_get_case_size (default_dict);
 
-  /* Create vfm_source. */
-  vfm_source = create_case_source (&input_program_source_class, inp);
+  proc_set_source (create_case_source (&input_program_source_class, inp));
 
   return CMD_SUCCESS;
 }
@@ -248,38 +250,12 @@
   inp->wc_data = wc_data;
   for (init_case (inp, c); ; clear_case (inp, c))
     {
-      int i;
-      
-      /* Perform transformations on `blank' case. */
-      for (i = 0; i < f_trns; )
-       {
-          int code;
-
-         code = t_trns[i].proc (t_trns[i].private, c, inp->case_nr);
-         switch (code)
-           {
-           case TRNS_CONTINUE:
-             i++;
-             break;
-
-            case TRNS_DROP_CASE:
-              break;
-
-            case TRNS_ERROR:
-              return false;
-
-           case TRNS_NEXT_CASE:
-             goto next_case;
-
-           case TRNS_END_FILE:
-              return true;
-
-           default:
-             i = code;
-             break;
-           }
-       }
-    next_case: ;
+      enum trns_result result = trns_chain_execute (inp->trns_chain, c,
+                                                    &inp->case_nr);
+      if (result == TRNS_ERROR)
+        return false;
+      else if (result == TRNS_END_FILE)
+        return true;
     }
 }
 
@@ -288,6 +264,7 @@
 {
   if (pgm != NULL) 
     {
+      trns_chain_destroy (pgm->trns_chain);
       free (pgm->init);
       free (pgm);
     }
Index: pspp/src/language/data-io/matrix-data.c
diff -u pspp/src/language/data-io/matrix-data.c:1.5 
pspp/src/language/data-io/matrix-data.c:1.6
--- pspp/src/language/data-io/matrix-data.c:1.5 Thu Apr 27 02:42:01 2006
+++ pspp/src/language/data-io/matrix-data.c     Thu May  4 06:19:02 2006
@@ -972,7 +972,8 @@
   nr.split_values = xnmalloc (dict_get_split_cnt (default_dict),
                               sizeof *nr.split_values);
 
-  vfm_source = create_case_source (&matrix_data_without_rowtype_source_class, 
&nr);
+  proc_set_source (create_case_source (
+                     &matrix_data_without_rowtype_source_class, &nr));
   
   ok = procedure (NULL, NULL);
 
@@ -1512,8 +1513,8 @@
   wr.current = NULL;
   mx->cells = 0;
 
-  vfm_source = create_case_source (&matrix_data_with_rowtype_source_class,
-                                   &wr);
+  proc_set_source (create_case_source (&matrix_data_with_rowtype_source_class,
+                                       &wr));
   ok = procedure (NULL, NULL);
 
   free (wr.split_values);
Index: pspp/src/language/data-io/print.c
diff -u pspp/src/language/data-io/print.c:1.5 
pspp/src/language/data-io/print.c:1.6
--- pspp/src/language/data-io/print.c:1.5       Tue Apr 25 20:13:05 2006
+++ pspp/src/language/data-io/print.c   Thu May  4 06:19:02 2006
@@ -20,21 +20,25 @@
 /* FIXME: seems like a lot of code duplication with data-list.c. */
 
 #include <config.h>
-#include <libpspp/message.h>
+
 #include <stdlib.h>
-#include <libpspp/alloc.h>
+
 #include <data/case.h>
+#include <procedure.h>
+#include <data/transformations.h>
+#include <data/variable.h>
 #include <language/command.h>
-#include <libpspp/compiler.h>
 #include <language/data-io/data-writer.h>
-#include <libpspp/message.h>
-#include <language/expressions/public.h>
 #include <language/data-io/file-handle.h>
+#include <language/expressions/public.h>
 #include <language/lexer/lexer.h>
+#include <libpspp/alloc.h>
+#include <libpspp/compiler.h>
+#include <libpspp/message.h>
+#include <libpspp/message.h>
 #include <libpspp/misc.h>
 #include <output/manager.h>
 #include <output/table.h>
-#include <data/variable.h>
 
 #include "gettext.h"
 #define _(msgid) gettext (msgid)
Index: pspp/src/language/dictionary/apply-dictionary.c
diff -u pspp/src/language/dictionary/apply-dictionary.c:1.4 
pspp/src/language/dictionary/apply-dictionary.c:1.5
--- pspp/src/language/dictionary/apply-dictionary.c:1.4 Sun Apr 16 04:46:41 2006
+++ pspp/src/language/dictionary/apply-dictionary.c     Thu May  4 06:19:02 2006
@@ -18,18 +18,21 @@
    02110-1301, USA. */
 
 #include <config.h>
+
 #include <stdlib.h>
+
 #include <data/any-reader.h>
-#include <language/command.h>
 #include <data/dictionary.h>
-#include <libpspp/message.h>
-#include <language/data-io/file-handle.h>
 #include <data/file-handle-def.h>
-#include <libpspp/hash.h>
-#include <language/lexer/lexer.h>
-#include <libpspp/str.h>
+#include <procedure.h>
 #include <data/value-labels.h>
 #include <data/variable.h>
+#include <language/command.h>
+#include <language/data-io/file-handle.h>
+#include <language/lexer/lexer.h>
+#include <libpspp/hash.h>
+#include <libpspp/message.h>
+#include <libpspp/str.h>
 
 #include "gettext.h"
 #define _(msgid) gettext (msgid)
Index: pspp/src/language/dictionary/formats.c
diff -u pspp/src/language/dictionary/formats.c:1.4 
pspp/src/language/dictionary/formats.c:1.5
--- pspp/src/language/dictionary/formats.c:1.4  Wed Apr 26 20:54:08 2006
+++ pspp/src/language/dictionary/formats.c      Thu May  4 06:19:02 2006
@@ -18,15 +18,18 @@
    02110-1301, USA. */
 
 #include <config.h>
+
 #include <limits.h>
 #include <stdio.h>
 #include <stdlib.h>
+
+#include <procedure.h>
+#include <data/variable.h>
 #include <language/command.h>
-#include <libpspp/message.h>
 #include <language/lexer/lexer.h>
+#include <libpspp/message.h>
 #include <libpspp/misc.h>
 #include <libpspp/str.h>
-#include <data/variable.h>
 
 #include "gettext.h"
 #define _(msgid) gettext (msgid)
Index: pspp/src/language/dictionary/missing-values.c
diff -u pspp/src/language/dictionary/missing-values.c:1.4 
pspp/src/language/dictionary/missing-values.c:1.5
--- pspp/src/language/dictionary/missing-values.c:1.4   Wed Apr 26 20:54:08 2006
+++ pspp/src/language/dictionary/missing-values.c       Thu May  4 06:19:02 2006
@@ -18,16 +18,19 @@
    02110-1301, USA. */
 
 #include <config.h>
-#include <libpspp/message.h>
+
 #include <stdlib.h>
-#include <language/command.h>
+
 #include <data/data-in.h>
-#include <libpspp/message.h>
+#include <procedure.h>
+#include <data/variable.h>
+#include <language/command.h>
 #include <language/lexer/lexer.h>
-#include <libpspp/magic.h>
 #include <language/lexer/range-parser.h>
+#include <libpspp/magic.h>
+#include <libpspp/message.h>
+#include <libpspp/message.h>
 #include <libpspp/str.h>
-#include <data/variable.h>
 
 #include "gettext.h"
 #define _(msgid) gettext (msgid)
Index: pspp/src/language/dictionary/modify-variables.c
diff -u pspp/src/language/dictionary/modify-variables.c:1.3 
pspp/src/language/dictionary/modify-variables.c:1.4
--- pspp/src/language/dictionary/modify-variables.c:1.3 Wed Mar 15 03:29:10 2006
+++ pspp/src/language/dictionary/modify-variables.c     Thu May  4 06:19:02 2006
@@ -88,12 +88,9 @@
 
   size_t i;
 
-  if (temporary != 0)
-    {
-      msg (SE, _("MODIFY VARS may not be used after TEMPORARY.  "
-                 "Temporary transformations will be made permanent."));
-      cancel_temporary (); 
-    }
+  if (proc_make_temporary_transformations_permanent ())
+    msg (SE, _("MODIFY VARS may not be used after TEMPORARY.  "
+               "Temporary transformations will be made permanent."));
 
   vm.reorder_vars = NULL;
   vm.reorder_cnt = 0;
Index: pspp/src/language/dictionary/numeric.c
diff -u pspp/src/language/dictionary/numeric.c:1.6 
pspp/src/language/dictionary/numeric.c:1.7
--- pspp/src/language/dictionary/numeric.c:1.6  Thu Apr 27 03:01:57 2006
+++ pspp/src/language/dictionary/numeric.c      Thu May  4 06:19:02 2006
@@ -18,14 +18,17 @@
    02110-1301, USA. */
 
 #include <config.h>
-#include <libpspp/message.h>
+
 #include <stdlib.h>
-#include <language/command.h>
+
 #include <data/dictionary.h>
-#include <libpspp/message.h>
+#include <procedure.h>
+#include <data/variable.h>
+#include <language/command.h>
 #include <language/lexer/lexer.h>
+#include <libpspp/message.h>
+#include <libpspp/message.h>
 #include <libpspp/str.h>
-#include <data/variable.h>
 
 #include "gettext.h"
 #define _(msgid) gettext (msgid)
Index: pspp/src/language/dictionary/rename-variables.c
diff -u pspp/src/language/dictionary/rename-variables.c:1.2 
pspp/src/language/dictionary/rename-variables.c:1.3
--- pspp/src/language/dictionary/rename-variables.c:1.2 Wed Mar 15 03:29:10 2006
+++ pspp/src/language/dictionary/rename-variables.c     Thu May  4 06:19:02 2006
@@ -18,16 +18,18 @@
    02110-1301, USA. */
 
 #include <config.h>
+
 #include <stdlib.h>
-#include <libpspp/message.h>
-#include <libpspp/alloc.h>
-#include <language/command.h>
+
 #include <data/dictionary.h>
-#include <libpspp/message.h>
-#include <libpspp/hash.h>
+#include <procedure.h>
+#include <data/variable.h>
+#include <language/command.h>
 #include <language/lexer/lexer.h>
+#include <libpspp/alloc.h>
+#include <libpspp/hash.h>
+#include <libpspp/message.h>
 #include <libpspp/str.h>
-#include <data/variable.h>
 
 #include "gettext.h"
 #define _(msgid) gettext (msgid)
@@ -44,12 +46,9 @@
 
   int status = CMD_CASCADING_FAILURE;
 
-  if (temporary != 0)
-    {
-      msg (SE, _("RENAME VARS may not be used after TEMPORARY.  "
-                 "Temporary transformations will be made permanent."));
-      cancel_temporary (); 
-    }
+  if (proc_make_temporary_transformations_permanent ())
+    msg (SE, _("RENAME VARS may not be used after TEMPORARY.  "
+               "Temporary transformations will be made permanent."));
 
   do
     {
Index: pspp/src/language/dictionary/split-file.c
diff -u pspp/src/language/dictionary/split-file.c:1.2 
pspp/src/language/dictionary/split-file.c:1.3
--- pspp/src/language/dictionary/split-file.c:1.2       Wed Mar 15 03:29:10 2006
+++ pspp/src/language/dictionary/split-file.c   Thu May  4 06:19:02 2006
@@ -18,14 +18,17 @@
    02110-1301, USA. */
 
 #include <config.h>
+
 #include <stdlib.h>
-#include <libpspp/alloc.h>
-#include <language/command.h>
+
 #include <data/dictionary.h>
-#include <libpspp/message.h>
+#include <procedure.h>
+#include <data/variable.h>
+#include <language/command.h>
 #include <language/lexer/lexer.h>
+#include <libpspp/alloc.h>
+#include <libpspp/message.h>
 #include <libpspp/str.h>
-#include <data/variable.h>
 
 int
 cmd_split_file (void)
Index: pspp/src/language/dictionary/sys-file-info.c
diff -u pspp/src/language/dictionary/sys-file-info.c:1.5 
pspp/src/language/dictionary/sys-file-info.c:1.6
--- pspp/src/language/dictionary/sys-file-info.c:1.5    Mon Apr 17 01:54:15 2006
+++ pspp/src/language/dictionary/sys-file-info.c        Thu May  4 06:19:02 2006
@@ -18,26 +18,29 @@
    02110-1301, USA. */
 
 #include <config.h>
-#include <libpspp/message.h>
+
 #include <ctype.h>
 #include <stdlib.h>
-#include <libpspp/array.h>
-#include <libpspp/alloc.h>
-#include <language/command.h>
+
 #include <data/dictionary.h>
-#include <libpspp/message.h>
 #include <data/file-handle-def.h>
+#include <procedure.h>
+#include <data/sys-file-reader.h>
+#include <data/value-labels.h>
+#include <data/variable.h>
+#include <language/command.h>
 #include <language/data-io/file-handle.h>
-#include <libpspp/hash.h>
 #include <language/lexer/lexer.h>
+#include <libpspp/alloc.h>
+#include <libpspp/array.h>
+#include <libpspp/hash.h>
 #include <libpspp/magic.h>
+#include <libpspp/message.h>
+#include <libpspp/message.h>
 #include <libpspp/misc.h>
-#include <output/output.h>
-#include <data/sys-file-reader.h>
 #include <output/manager.h>
+#include <output/output.h>
 #include <output/table.h>
-#include <data/value-labels.h>
-#include <data/variable.h>
 
 #include "gettext.h"
 #define _(msgid) gettext (msgid)
Index: pspp/src/language/dictionary/value-labels.c
diff -u pspp/src/language/dictionary/value-labels.c:1.3 
pspp/src/language/dictionary/value-labels.c:1.4
--- pspp/src/language/dictionary/value-labels.c:1.3     Wed Apr 26 20:54:08 2006
+++ pspp/src/language/dictionary/value-labels.c Thu May  4 06:19:02 2006
@@ -18,16 +18,19 @@
    02110-1301, USA. */
 
 #include <config.h>
+
 #include <stdio.h>
 #include <stdlib.h>
-#include <libpspp/alloc.h>
+
+#include <procedure.h>
+#include <data/value-labels.h>
+#include <data/variable.h>
 #include <language/command.h>
-#include <libpspp/message.h>
-#include <libpspp/hash.h>
 #include <language/lexer/lexer.h>
+#include <libpspp/alloc.h>
+#include <libpspp/hash.h>
+#include <libpspp/message.h>
 #include <libpspp/str.h>
-#include <data/value-labels.h>
-#include <data/variable.h>
 
 #include "gettext.h"
 #define _(msgid) gettext (msgid)
Index: pspp/src/language/dictionary/variable-display.c
diff -u pspp/src/language/dictionary/variable-display.c:1.4 
pspp/src/language/dictionary/variable-display.c:1.5
--- pspp/src/language/dictionary/variable-display.c:1.4 Wed Apr 26 20:54:08 2006
+++ pspp/src/language/dictionary/variable-display.c     Thu May  4 06:19:02 2006
@@ -18,14 +18,17 @@
    02110-1301, USA. */
 
 #include <config.h>
+
 #include <stdio.h>
 #include <stdlib.h>
-#include <libpspp/alloc.h>
+
+#include <procedure.h>
+#include <data/variable.h>
 #include <language/command.h>
-#include <libpspp/message.h>
 #include <language/lexer/lexer.h>
+#include <libpspp/alloc.h>
+#include <libpspp/message.h>
 #include <libpspp/str.h>
-#include <data/variable.h>
 
 /* Set variables' alignment
    This is the alignment for GUI display only.
@@ -42,7 +45,6 @@
       size_t i;
       enum alignment align;
 
-
       if (!parse_variables (default_dict, &v, &nv, PV_NONE))
         return CMD_FAILURE;
 
Index: pspp/src/language/dictionary/variable-label.c
diff -u pspp/src/language/dictionary/variable-label.c:1.4 
pspp/src/language/dictionary/variable-label.c:1.5
--- pspp/src/language/dictionary/variable-label.c:1.4   Wed Apr 26 20:54:08 2006
+++ pspp/src/language/dictionary/variable-label.c       Thu May  4 06:19:02 2006
@@ -18,14 +18,17 @@
    02110-1301, USA. */
 
 #include <config.h>
+
 #include <stdio.h>
 #include <stdlib.h>
-#include <libpspp/alloc.h>
+
+#include <procedure.h>
+#include <data/variable.h>
 #include <language/command.h>
-#include <libpspp/message.h>
 #include <language/lexer/lexer.h>
+#include <libpspp/alloc.h>
+#include <libpspp/message.h>
 #include <libpspp/str.h>
-#include <data/variable.h>
 
 #include "gettext.h"
 #define _(msgid) gettext (msgid)
Index: pspp/src/language/dictionary/vector.c
diff -u pspp/src/language/dictionary/vector.c:1.3 
pspp/src/language/dictionary/vector.c:1.4
--- pspp/src/language/dictionary/vector.c:1.3   Wed Apr 26 20:54:08 2006
+++ pspp/src/language/dictionary/vector.c       Thu May  4 06:19:02 2006
@@ -18,16 +18,18 @@
    02110-1301, USA. */
 
 #include <config.h>
-#include <libpspp/message.h>
+
 #include <stdlib.h>
-#include <libpspp/alloc.h>
-#include <language/command.h>
+
+#include <procedure.h>
 #include <data/dictionary.h>
-#include <libpspp/message.h>
+#include <data/variable.h>
+#include <language/command.h>
 #include <language/lexer/lexer.h>
+#include <libpspp/alloc.h>
+#include <libpspp/message.h>
 #include <libpspp/misc.h>
 #include <libpspp/str.h>
-#include <data/variable.h>
 
 #include "gettext.h"
 #define _(msgid) gettext (msgid)
Index: pspp/src/language/dictionary/weight.c
diff -u pspp/src/language/dictionary/weight.c:1.2 
pspp/src/language/dictionary/weight.c:1.3
--- pspp/src/language/dictionary/weight.c:1.2   Wed Mar 15 03:29:10 2006
+++ pspp/src/language/dictionary/weight.c       Thu May  4 06:19:02 2006
@@ -18,14 +18,16 @@
    02110-1301, USA. */
 
 #include <config.h>
-#include <libpspp/message.h>
+
 #include <stdio.h>
-#include <language/command.h>
+
+#include <procedure.h>
 #include <data/dictionary.h>
-#include <libpspp/message.h>
+#include <data/variable.h>
+#include <language/command.h>
 #include <language/lexer/lexer.h>
+#include <libpspp/message.h>
 #include <libpspp/str.h>
-#include <data/variable.h>
 
 #include "gettext.h"
 #define _(msgid) gettext (msgid)
Index: pspp/src/language/expressions/parse.c
diff -u pspp/src/language/expressions/parse.c:1.6 
pspp/src/language/expressions/parse.c:1.7
--- pspp/src/language/expressions/parse.c:1.6   Tue May  2 03:51:43 2006
+++ pspp/src/language/expressions/parse.c       Thu May  4 06:19:02 2006
@@ -1222,7 +1222,7 @@
       msg (SE, _("%s is not yet implemented."), f->prototype);
       goto fail;
     }
-  if ((f->flags & OPF_PERM_ONLY) && temporary != 0) 
+  if ((f->flags & OPF_PERM_ONLY) && proc_in_temporary_transformations ()) 
     {
       msg (SE, _("%s may not appear after TEMPORARY."), f->prototype);
       goto fail;
Index: pspp/src/language/lexer/variable-parser.c
diff -u pspp/src/language/lexer/variable-parser.c:1.2 
pspp/src/language/lexer/variable-parser.c:1.3
--- pspp/src/language/lexer/variable-parser.c:1.2       Wed Mar 15 03:29:11 2006
+++ pspp/src/language/lexer/variable-parser.c   Thu May  4 06:19:02 2006
@@ -18,19 +18,21 @@
    02110-1301, USA. */
 
 #include <config.h>
-#include <data/variable.h>
+
 #include <ctype.h>
 #include <stdbool.h>
 #include <stdlib.h>
+
+#include "lexer.h"
+#include <data/dictionary.h>
+#include <procedure.h>
+#include <data/variable.h>
 #include <libpspp/alloc.h>
 #include <libpspp/bit-vector.h>
-#include <data/dictionary.h>
-#include <libpspp/message.h>
 #include <libpspp/hash.h>
-#include "lexer.h"
+#include <libpspp/message.h>
 #include <libpspp/misc.h>
 #include <libpspp/pool.h>
-#include "size_max.h"
 #include <libpspp/str.h>
 
 #include "gettext.h"
Index: pspp/src/language/stats/ChangeLog
diff -u pspp/src/language/stats/ChangeLog:1.17 
pspp/src/language/stats/ChangeLog:1.18
--- pspp/src/language/stats/ChangeLog:1.17      Fri Apr 28 19:17:15 2006
+++ pspp/src/language/stats/ChangeLog   Thu May  4 06:19:02 2006
@@ -1,3 +1,14 @@
+Wed May  3 23:05:31 2006  Ben Pfaff  <address@hidden>
+
+       Continue reforming procedure execution.  In this phase, get rid of
+       many global variables, consolidating procedure execution in
+       procedure.c.  Encapsulate transformations in new "struct
+       trns_chain".  Also, change implementation of N OF CASES, FILTER,
+       and PROCESS IF from special cases to transformations.
+        
+       * aggregate.c (cmd_aggregate) Use discard_variables().
+       
+
 2006-04-28  Jason Stover  <address@hidden>
 
        * regression.q (regression_trns_resid_proc): Pass only the
Index: pspp/src/language/stats/aggregate.c
diff -u pspp/src/language/stats/aggregate.c:1.5 
pspp/src/language/stats/aggregate.c:1.6
--- pspp/src/language/stats/aggregate.c:1.5     Thu Apr 27 02:42:01 2006
+++ pspp/src/language/stats/aggregate.c Thu May  4 06:19:02 2006
@@ -27,6 +27,7 @@
 #include <data/casefile.h>
 #include <data/dictionary.h>
 #include <data/file-handle-def.h>
+#include <procedure.h>
 #include <data/settings.h>
 #include <data/storage-stream.h>
 #include <data/sys-file-writer.h>
@@ -260,7 +261,7 @@
     {
       /* The active file will be replaced by the aggregated data,
          so TEMPORARY is moot. */
-      cancel_temporary ();
+      proc_cancel_temporary_transformations ();
 
       if (agr.sort != NULL && !presorted) 
         {
@@ -271,7 +272,7 @@
       agr.sink = create_case_sink (&storage_sink_class, agr.dict, NULL);
       if (agr.sink->class->open != NULL)
         agr.sink->class->open (agr.sink);
-      vfm_sink = create_case_sink (&null_sink_class, default_dict, NULL);
+      proc_set_sink (create_case_sink (&null_sink_class, default_dict, NULL));
       if (!procedure (agr_to_active_file, &agr))
         goto error;
       if (agr.case_cnt > 0) 
@@ -280,10 +281,10 @@
           if (!agr.sink->class->write (agr.sink, &agr.agr_case))
             goto error;
         }
-      dict_destroy (default_dict);
+      discard_variables ();
       default_dict = agr.dict;
       agr.dict = NULL;
-      vfm_source = agr.sink->class->make_source (agr.sink);
+      proc_set_source (agr.sink->class->make_source (agr.sink));
       free_case_sink (agr.sink);
     }
   else
Index: pspp/src/language/stats/autorecode.c
diff -u pspp/src/language/stats/autorecode.c:1.4 
pspp/src/language/stats/autorecode.c:1.5
--- pspp/src/language/stats/autorecode.c:1.4    Wed Apr 26 22:00:29 2006
+++ pspp/src/language/stats/autorecode.c        Thu May  4 06:19:02 2006
@@ -18,19 +18,21 @@
    02110-1301, USA. */
 
 #include <config.h>
-#include <libpspp/message.h>
 #include <stdlib.h>
-#include <libpspp/alloc.h>
+
 #include <data/case.h>
+#include <data/dictionary.h>
+#include <data/transformations.h>
+#include <data/variable.h>
 #include <language/command.h>
+#include <language/lexer/lexer.h>
+#include <libpspp/alloc.h>
 #include <libpspp/compiler.h>
-#include <data/dictionary.h>
-#include <libpspp/message.h>
 #include <libpspp/hash.h>
-#include <language/lexer/lexer.h>
+#include <libpspp/message.h>
+#include <libpspp/message.h>
 #include <libpspp/pool.h>
 #include <libpspp/str.h>
-#include <data/variable.h>
 #include <procedure.h>
 
 #include "gettext.h"
Index: pspp/src/language/stats/correlations.q
diff -u pspp/src/language/stats/correlations.q:1.5 
pspp/src/language/stats/correlations.q:1.6
--- pspp/src/language/stats/correlations.q:1.5  Sun Apr 16 04:46:41 2006
+++ pspp/src/language/stats/correlations.q      Thu May  4 06:19:02 2006
@@ -18,15 +18,19 @@
    02110-1301, USA. */
 
 #include <config.h>
+
 #include <stdlib.h>
-#include <libpspp/alloc.h>
-#include <libpspp/compiler.h>
+
 #include <data/dictionary.h>
 #include <data/file-handle-def.h>
-#include <language/data-io/file-handle.h>
+#include <procedure.h>
+#include <data/variable.h>
 #include <language/command.h>
+#include <language/data-io/file-handle.h>
 #include <language/lexer/lexer.h>
-#include <data/variable.h>
+#include <libpspp/alloc.h>
+#include <libpspp/compiler.h>
+
 /* (headers) */
 
 struct cor_set
Index: pspp/src/language/stats/descriptives.c
diff -u pspp/src/language/stats/descriptives.c:1.5 
pspp/src/language/stats/descriptives.c:1.6
--- pspp/src/language/stats/descriptives.c:1.5  Wed Apr 26 22:00:29 2006
+++ pspp/src/language/stats/descriptives.c      Thu May  4 06:19:02 2006
@@ -20,24 +20,26 @@
 /* FIXME: Many possible optimizations. */
 
 #include <config.h>
-#include <libpspp/message.h>
+
 #include <limits.h>
 #include <math.h>
 #include <stdlib.h>
-#include <libpspp/array.h>
-#include <libpspp/alloc.h>
+
 #include <data/case.h>
 #include <data/casefile.h>
-#include <language/command.h>
-#include <libpspp/compiler.h>
 #include <data/dictionary.h>
+#include <data/transformations.h>
+#include <data/variable.h>
+#include <language/command.h>
 #include <language/lexer/lexer.h>
-#include <libpspp/message.h>
+#include <libpspp/alloc.h>
+#include <libpspp/array.h>
+#include <libpspp/compiler.h>
 #include <libpspp/magic.h>
+#include <libpspp/message.h>
 #include <math/moments.h>
 #include <output/manager.h>
 #include <output/table.h>
-#include <data/variable.h>
 #include <procedure.h>
 
 #include "gettext.h"
Index: pspp/src/language/stats/flip.c
diff -u pspp/src/language/stats/flip.c:1.6 pspp/src/language/stats/flip.c:1.7
--- pspp/src/language/stats/flip.c:1.6  Thu Apr 27 02:42:01 2006
+++ pspp/src/language/stats/flip.c      Thu May  4 06:19:02 2006
@@ -32,6 +32,7 @@
 #include <data/case-source.h>
 #include <data/case.h>
 #include <data/dictionary.h>
+#include <procedure.h>
 #include <data/settings.h>
 #include <data/value.h>
 #include <data/variable.h>
@@ -91,14 +92,12 @@
 cmd_flip (void)
 {
   struct flip_pgm *flip;
+  struct case_sink *sink;
   bool ok;
 
-  if (temporary != 0)
-    {
-      msg (SW, _("FLIP ignores TEMPORARY.  "
-                 "Temporary transformations will be made permanent."));
-      cancel_temporary (); 
-    }
+  if (proc_make_temporary_transformations_permanent ())
+    msg (SW, _("FLIP ignores TEMPORARY.  "
+               "Temporary transformations will be made permanent."));
 
   flip = pool_create_container (struct flip_pgm, pool);
   flip->var = NULL;
@@ -150,10 +149,11 @@
 
   /* Read the active file into a flip_sink. */
   flip->case_cnt = 0;
-  temp_trns = temporary = 0;
-  vfm_sink = flip_sink_create (flip);
-  if (vfm_sink == NULL)
+  proc_make_temporary_transformations_permanent ();
+  sink = flip_sink_create (flip);
+  if (sink == NULL)
     goto error;
+  proc_set_sink (sink);
   flip->new_names_tail = NULL;
   ok = procedure (NULL, NULL);
 
@@ -174,7 +174,7 @@
   flip->case_size = dict_get_case_size (default_dict);
 
   /* Set up flipped data for reading. */
-  vfm_source = flip_source_create (flip);
+  proc_set_source (flip_source_create (flip));
 
   return ok ? lex_end_of_command () : CMD_CASCADING_FAILURE;
 
Index: pspp/src/language/stats/means.q
diff -u pspp/src/language/stats/means.q:1.3 pspp/src/language/stats/means.q:1.4
--- pspp/src/language/stats/means.q:1.3 Sun Apr 16 04:46:41 2006
+++ pspp/src/language/stats/means.q     Thu May  4 06:19:02 2006
@@ -18,17 +18,19 @@
    02110-1301, USA. */
 
 #include <config.h>
+
 #include <stdlib.h>
 #include <stdio.h>
+
 #include <data/dictionary.h>
-#include <libpspp/message.h>
-#include <libpspp/alloc.h>
+#include <procedure.h>
+#include <data/variable.h>
 #include <language/command.h>
-#include <libpspp/hash.h>
 #include <language/lexer/lexer.h>
-#include <libpspp/message.h>
+#include <libpspp/alloc.h>
+#include <libpspp/hash.h>
 #include <libpspp/magic.h>
-#include <data/variable.h>
+#include <libpspp/message.h>
 
 #include "gettext.h"
 #define _(msgid) gettext (msgid)
Index: pspp/src/language/stats/rank.q
diff -u pspp/src/language/stats/rank.q:1.6 pspp/src/language/stats/rank.q:1.7
--- pspp/src/language/stats/rank.q:1.6  Tue Apr 25 20:19:13 2006
+++ pspp/src/language/stats/rank.q      Thu May  4 06:19:02 2006
@@ -19,14 +19,15 @@
 02110-1301, USA. */
 
 #include <config.h>
+
+#include "sort-criteria.h"
+#include <data/dictionary.h>
+#include <procedure.h>
+#include <data/variable.h>
 #include <language/command.h>
 #include <language/stats/sort-criteria.h>
 #include <libpspp/compiler.h>
-#include <data/dictionary.h>
 #include <math/sort.h>
-#include <data/variable.h>
-
-#include "sort-criteria.h"
 
 #include "gettext.h"
 #define _(msgid) gettext (msgid)
Index: pspp/src/language/stats/regression.q
diff -u pspp/src/language/stats/regression.q:1.20 
pspp/src/language/stats/regression.q:1.21
--- pspp/src/language/stats/regression.q:1.20   Fri Apr 28 19:17:15 2006
+++ pspp/src/language/stats/regression.q        Thu May  4 06:19:02 2006
@@ -18,33 +18,37 @@
    02110-1301, USA. */
 
 #include <config.h>
-#include <stdlib.h>
+
 #include <gsl/gsl_cdf.h>
-#include <gsl/gsl_vector.h>
 #include <gsl/gsl_matrix.h>
+#include <gsl/gsl_vector.h>
 #include <math.h>
-#include <libpspp/alloc.h>
+#include <stdlib.h>
+
+#include "regression-export.h"
 #include <data/case.h>
 #include <data/casefile.h>
-#include <data/category.h>
 #include <data/cat-routines.h>
-#include <language/command.h>
-#include <libpspp/compiler.h>
-#include <math/design-matrix.h>
+#include <data/category.h>
 #include <data/dictionary.h>
-#include <libpspp/message.h>
+#include <data/missing-values.h>
+#include <data/transformations.h>
+#include <data/value-labels.h>
+#include <data/variable.h>
+#include <language/command.h>
 #include <language/data-io/file-handle.h>
-#include "gettext.h"
 #include <language/lexer/lexer.h>
-#include <math/linreg/linreg.h>
+#include <libpspp/alloc.h>
+#include <libpspp/compiler.h>
+#include <libpspp/message.h>
+#include <math/design-matrix.h>
 #include <math/linreg/coefficient.h>
-#include <data/missing-values.h>
-#include "regression-export.h"
+#include <math/linreg/linreg.h>
 #include <output/table.h>
-#include <data/value-labels.h>
-#include <data/variable.h>
 #include <procedure.h>
 
+#include "gettext.h"
+
 #define REG_LARGE_DATA 1000
 
 /* (headers) */
Index: pspp/src/language/stats/sort-cases.c
diff -u pspp/src/language/stats/sort-cases.c:1.2 
pspp/src/language/stats/sort-cases.c:1.3
--- pspp/src/language/stats/sort-cases.c:1.2    Wed Mar 15 03:29:11 2006
+++ pspp/src/language/stats/sort-cases.c        Thu May  4 06:19:02 2006
@@ -18,18 +18,21 @@
    02110-1301, USA. */
 
 #include <config.h>
-#include <sys/types.h>
+
 #include <assert.h>
 #include <stdlib.h>
 #include <limits.h>
-#include <libpspp/alloc.h>
+
+#include "sort-criteria.h"
+#include <procedure.h>
+#include <data/settings.h>
+#include <data/variable.h>
 #include <language/command.h>
-#include <libpspp/message.h>
 #include <language/lexer/lexer.h>
-#include <data/settings.h>
-#include "sort-criteria.h"
+#include <libpspp/alloc.h>
+#include <libpspp/message.h>
 #include <math/sort.h>
-#include <data/variable.h>
+#include <sys/types.h>
 
 #include "gettext.h"
 #define _(msgid) gettext (msgid)
Index: pspp/src/language/utilities/set.q
diff -u pspp/src/language/utilities/set.q:1.6 
pspp/src/language/utilities/set.q:1.7
--- pspp/src/language/utilities/set.q:1.6       Wed Apr 26 22:06:48 2006
+++ pspp/src/language/utilities/set.q   Thu May  4 06:19:02 2006
@@ -18,27 +18,27 @@
    02110-1301, USA. */
 
 #include <config.h>
-#include <data/settings.h>
-#include <libpspp/message.h>
+
 #include <stdio.h>
 #include <errno.h>
 #include <stdlib.h>
 #include <time.h>
-#include <libpspp/alloc.h>
-#include <language/command.h>
-#include <libpspp/compiler.h>
+
 #include <data/dictionary.h>
-#include <language/line-buffer.h>
+#include <data/format.h>
+#include <procedure.h>
+#include <data/settings.h>
+#include <data/variable.h>
+#include <language/command.h>
 #include <language/lexer/lexer.h>
-#include <libpspp/message.h>
+#include <language/line-buffer.h>
+#include <libpspp/alloc.h>
+#include <libpspp/compiler.h>
+#include <libpspp/copyleft.h>
 #include <libpspp/magic.h>
-#include <output/output.h>
+#include <libpspp/message.h>
 #include <math/random.h>
-#include <data/variable.h>
-#include <data/format.h>
-#include <libpspp/copyleft.h>
-#include <data/variable.h>
-
+#include <output/output.h>
 
 #if HAVE_LIBTERMCAP
 #if HAVE_TERMCAP_H
Index: pspp/src/language/xforms/compute.c
diff -u pspp/src/language/xforms/compute.c:1.3 
pspp/src/language/xforms/compute.c:1.4
--- pspp/src/language/xforms/compute.c:1.3      Thu Apr 27 03:01:57 2006
+++ pspp/src/language/xforms/compute.c  Thu May  4 06:19:02 2006
@@ -18,18 +18,22 @@
    02110-1301, USA. */
 
 #include <config.h>
-#include <libpspp/message.h>
+
 #include <stdlib.h>
-#include <libpspp/alloc.h>
+
 #include <data/case.h>
-#include <language/command.h>
 #include <data/dictionary.h>
-#include <libpspp/message.h>
+#include <procedure.h>
+#include <data/transformations.h>
+#include <data/variable.h>
+#include <language/command.h>
 #include <language/expressions/public.h>
 #include <language/lexer/lexer.h>
+#include <libpspp/alloc.h>
+#include <libpspp/message.h>
+#include <libpspp/message.h>
 #include <libpspp/misc.h>
 #include <libpspp/str.h>
-#include <data/variable.h>
 
 #include "gettext.h"
 #define _(msgid) gettext (msgid)
Index: pspp/src/language/xforms/count.c
diff -u pspp/src/language/xforms/count.c:1.3 
pspp/src/language/xforms/count.c:1.4
--- pspp/src/language/xforms/count.c:1.3        Wed Mar 15 03:29:11 2006
+++ pspp/src/language/xforms/count.c    Thu May  4 06:19:02 2006
@@ -18,19 +18,23 @@
    02110-1301, USA. */
 
 #include <config.h>
-#include <libpspp/message.h>
+
 #include <stdlib.h>
-#include <libpspp/alloc.h>
+
 #include <data/case.h>
+#include <data/dictionary.h>
+#include <procedure.h>
+#include <data/transformations.h>
+#include <data/variable.h>
 #include <language/command.h>
+#include <language/lexer/lexer.h>
+#include <language/lexer/range-parser.h>
+#include <libpspp/alloc.h>
 #include <libpspp/compiler.h>
-#include <data/dictionary.h>
 #include <libpspp/message.h>
-#include <language/lexer/lexer.h>
+#include <libpspp/message.h>
 #include <libpspp/pool.h>
-#include <language/lexer/range-parser.h>
 #include <libpspp/str.h>
-#include <data/variable.h>
 
 #include "gettext.h"
 #define _(msgid) gettext (msgid)
Index: pspp/src/language/xforms/recode.c
diff -u pspp/src/language/xforms/recode.c:1.4 
pspp/src/language/xforms/recode.c:1.5
--- pspp/src/language/xforms/recode.c:1.4       Wed Apr 26 20:54:08 2006
+++ pspp/src/language/xforms/recode.c   Thu May  4 06:19:02 2006
@@ -18,23 +18,27 @@
    02110-1301, USA. */
 
 #include <config.h>
-#include <libpspp/message.h>
+
 #include <ctype.h>
 #include <math.h>
 #include <stdlib.h>
-#include <libpspp/alloc.h>
+
 #include <data/case.h>
-#include <language/command.h>
-#include <libpspp/compiler.h>
 #include <data/data-in.h>
 #include <data/dictionary.h>
-#include <libpspp/message.h>
+#include <procedure.h>
+#include <data/transformations.h>
+#include <data/variable.h>
+#include <language/command.h>
 #include <language/lexer/lexer.h>
+#include <language/lexer/range-parser.h>
+#include <libpspp/alloc.h>
+#include <libpspp/compiler.h>
 #include <libpspp/magic.h>
+#include <libpspp/message.h>
+#include <libpspp/message.h>
 #include <libpspp/pool.h>
-#include <language/lexer/range-parser.h>
 #include <libpspp/str.h>
-#include <data/variable.h>
 
 #include "gettext.h"
 #define _(msgid) gettext (msgid)
Index: pspp/src/language/xforms/sample.c
diff -u pspp/src/language/xforms/sample.c:1.4 
pspp/src/language/xforms/sample.c:1.5
--- pspp/src/language/xforms/sample.c:1.4       Sun Apr 16 04:46:41 2006
+++ pspp/src/language/xforms/sample.c   Thu May  4 06:19:02 2006
@@ -18,18 +18,21 @@
    02110-1301, USA. */
 
 #include <config.h>
+
 #include <gsl/gsl_rng.h>
 #include <limits.h>
 #include <stdio.h>
 #include <math.h>
-#include <libpspp/alloc.h>
+
+#include <data/variable.h>
 #include <language/command.h>
+#include <language/lexer/lexer.h>
+#include <libpspp/alloc.h>
 #include <libpspp/compiler.h>
 #include <libpspp/message.h>
-#include <language/lexer/lexer.h>
-#include <math/random.h>
 #include <libpspp/str.h>
-#include <data/variable.h>
+#include <math/random.h>
+#include <procedure.h>
 
 #include "gettext.h"
 #define _(msgid) gettext (msgid)
Index: pspp/src/language/xforms/select-if.c
diff -u pspp/src/language/xforms/select-if.c:1.3 
pspp/src/language/xforms/select-if.c:1.4
--- pspp/src/language/xforms/select-if.c:1.3    Tue Apr 25 20:24:11 2006
+++ pspp/src/language/xforms/select-if.c        Thu May  4 06:19:02 2006
@@ -18,15 +18,19 @@
    02110-1301, USA. */
 
 #include <config.h>
+
 #include <stdlib.h>
-#include <libpspp/alloc.h>
-#include <language/command.h>
+
 #include <data/dictionary.h>
-#include <libpspp/message.h>
+#include <data/transformations.h>
+#include <procedure.h>
+#include <data/variable.h>
+#include <language/command.h>
 #include <language/expressions/public.h>
 #include <language/lexer/lexer.h>
+#include <libpspp/alloc.h>
+#include <libpspp/message.h>
 #include <libpspp/str.h>
-#include <data/variable.h>
 
 #include "gettext.h"
 #define _(msgid) gettext (msgid)
Index: pspp/src/math/ChangeLog
diff -u pspp/src/math/ChangeLog:1.3 pspp/src/math/ChangeLog:1.4
--- pspp/src/math/ChangeLog:1.3 Sat Apr 29 03:12:03 2006
+++ pspp/src/math/ChangeLog     Thu May  4 06:19:02 2006
@@ -1,3 +1,21 @@
+Wed May  3 23:06:43 2006  Ben Pfaff  <address@hidden>
+
+       Continue reforming procedure execution.  In this phase, get rid of
+       many global variables, consolidating procedure execution in
+       procedure.c.  Encapsulate transformations in new "struct
+       trns_chain".  Also, change implementation of N OF CASES, FILTER,
+       and PROCESS IF from special cases to transformations.
+        
+       * sort.c: (prepare_to_sort_active_file) Don't run a procedure
+       here.
+       (sort_active_file_in_place) Rewrite to run a procedure, capture
+       the output, sort the output, and set that as the source for the
+       next procedure.
+       (struct sort_to_casefile_cb_data) New structure.
+       (sort_to_casefile_callback) New function.
+       (sort_active_file_to_casefile) Rewrite to use
+       multipass_procedure().
+
 Sat Apr 29 11:09:33 WST 2006 John Darrington <address@hidden>
 
        * removed unused variable.
Index: pspp/src/math/sort.c
diff -u pspp/src/math/sort.c:1.6 pspp/src/math/sort.c:1.7
--- pspp/src/math/sort.c:1.6    Sat Apr 29 03:12:03 2006
+++ pspp/src/math/sort.c        Thu May  4 06:19:02 2006
@@ -57,24 +57,13 @@
 static struct casefile *do_external_sort (struct casereader *,
                                           const struct sort_criteria *);
 
-/* Gets ready to sort the active file, either in-place or to a
-   separate casefile. */
-static bool
+/* Get ready to sort the active file. */
+static void
 prepare_to_sort_active_file (void) 
 {
-  bool ok;
-  
-  /* Cancel temporary transformations and PROCESS IF. */
-  if (temporary != 0)
-    cancel_temporary (); 
+  proc_cancel_temporary_transformations (); 
   expr_free (process_if_expr);
   process_if_expr = NULL;
-
-  /* Make sure source cases are in a storage source. */
-  ok = procedure (NULL, NULL);
-  assert (case_source_is_class (vfm_source, &storage_source_class));
-
-  return ok;
 }
 
 /* Sorts the active file in-place according to CRITERIA.
@@ -82,21 +71,35 @@
 int
 sort_active_file_in_place (const struct sort_criteria *criteria) 
 {
-  struct casefile *src, *dst;
+  struct casefile *in, *out;
+
+  prepare_to_sort_active_file ();
+  if (!procedure (NULL, NULL))
+    return 0;
   
-  if (!prepare_to_sort_active_file ())
+  in = proc_capture_output ();
+  out = sort_execute (casefile_get_destructive_reader (in), criteria);
+  if (out == NULL) 
     return 0;
 
-  src = storage_source_get_casefile (vfm_source);
-  dst = sort_execute (casefile_get_destructive_reader (src), criteria);
-  free_case_source (vfm_source);
-  vfm_source = NULL;
+  proc_set_source (storage_source_create (out));
+  return 1;
+}
 
-  if (dst == NULL) 
-    return 0;
+/* Data passed to sort_to_casefile_callback(). */
+struct sort_to_casefile_cb_data 
+  {
+    const struct sort_criteria *criteria;
+    struct casefile *output;
+  };
 
-  vfm_source = storage_source_create (dst);
-  return 1;
+/* Sorts casefile CF according to the criteria in CB_DATA. */
+static bool
+sort_to_casefile_callback (const struct casefile *cf, void *cb_data_) 
+{
+  struct sort_to_casefile_cb_data *cb_data = cb_data_;
+  cb_data->output = sort_execute (casefile_get_reader (cf), cb_data->criteria);
+  return cb_data->output != NULL;
 }
 
 /* Sorts the active file to a separate casefile.  If successful,
@@ -105,13 +108,15 @@
 struct casefile *
 sort_active_file_to_casefile (const struct sort_criteria *criteria) 
 {
-  struct casefile *src;
+  struct sort_to_casefile_cb_data cb_data;
   
-  if (!prepare_to_sort_active_file ())
-    return NULL;
+  prepare_to_sort_active_file ();
+
+  cb_data.criteria = criteria;
+  cb_data.output = NULL;
+  multipass_procedure (sort_to_casefile_callback, &cb_data);
 
-  src = storage_source_get_casefile (vfm_source);
-  return sort_execute (casefile_get_reader (src), criteria);
+  return cb_data.output;
 }
 
 
Index: pspp/src/procedure.c
diff -u pspp/src/procedure.c:1.12 pspp/src/procedure.c:1.13
--- pspp/src/procedure.c:1.12   Fri Apr 28 04:50:22 2006
+++ pspp/src/procedure.c        Thu May  4 06:19:02 2006
@@ -35,9 +35,9 @@
 #include <data/file-handle-def.h>
 #include <data/settings.h>
 #include <data/storage-stream.h>
+#include <data/transformations.h>
 #include <data/value-labels.h>
 #include <data/variable.h>
-#include <language/control/control-stack.h>
 #include <libpspp/alloc.h>
 #include <libpspp/message.h>
 #include <libpspp/misc.h>
@@ -71,32 +71,40 @@
     size_t cases_analyzed;      /* Cases passed to procedure so far. */
   };
 
-/* The current active file, from which cases are read. */
-struct case_source *vfm_source;
+/* Cases are read from vfm_source,
+   pass through permanent_trns_chain (which transforms them into
+   the format described by permanent_dict),
+   are written to vfm_sink,
+   pass through temporary_trns_chain (which transforms them into
+   the format described by default_dict),
+   and are finally passed to the procedure. */
+static struct case_source *vfm_source;
+static struct trns_chain *permanent_trns_chain;
+static struct dictionary *permanent_dict;
+static struct case_sink *vfm_sink;
+static struct trns_chain *temporary_trns_chain;
+struct dictionary *default_dict;
+
+/* The transformation chain that the next transformation will be
+   added to. */
+static struct trns_chain *cur_trns_chain;
 
-/* The replacement active file, to which cases are written. */
-struct case_sink *vfm_sink;
-
-/* The compactor used to compact a compact, if necessary;
+/* The compactor used to compact a case, if necessary;
    otherwise a null pointer. */
 static struct dict_compactor *compactor;
 
 /* Time at which vfm was last invoked. */
 static time_t last_vfm_invocation;
 
-/* Whether we're inside a procedure.
-   For debugging purposes only. */
-static bool in_procedure;
-
 /* Lag queue. */
 int n_lag;                     /* Number of cases to lag. */
 static int lag_count;          /* Number of cases in lag_queue so far. */
 static int lag_head;           /* Index where next case will be added. */
 static struct ccase *lag_queue; /* Array of n_lag ccase * elements. */
 
-/* Active transformations. */
-struct transformation *t_trns;
-size_t n_trns, m_trns, f_trns;
+static void add_case_limit_trns (void);
+static void add_filter_trns (void);
+static void add_process_if_trns (void);
 
 static bool internal_procedure (bool (*proc_func) (struct ccase *, void *),
                                 void *aux);
@@ -104,11 +112,6 @@
 static void create_trns_case (struct ccase *, struct dictionary *);
 static void open_active_file (void);
 static bool write_case (struct write_case_data *wc_data);
-static int execute_transformations (struct ccase *c,
-                                    struct transformation *trns,
-                                    int first_idx, int last_idx,
-                                    int case_num);
-static int filter_case (const struct ccase *c, int case_num);
 static void lag_case (const struct ccase *c);
 static void clear_case (struct ccase *c);
 static bool close_active_file (void);
@@ -126,26 +129,17 @@
 
 /* Reads the data from the input program and writes it to a new
    active file.  For each case we read from the input program, we
-   do the following
+   do the following:
 
    1. Execute permanent transformations.  If these drop the case,
       start the next case from step 1.
 
-   2. N OF CASES.  If we have already written N cases, start the
-      next case from step 1.
+   2. Write case to replacement active file.
    
-   3. Write case to replacement active file.
-   
-   4. Execute temporary transformations.  If these drop the case,
+   3. Execute temporary transformations.  If these drop the case,
       start the next case from step 1.
       
-   5. FILTER, PROCESS IF.  If these drop the case, start the next
-      case from step 1.
-   
-   6. Post-TEMPORARY N OF CASES.  If we have already analyzed N
-      cases, start the next case from step 1.
-      
-   7. Pass case to PROC_FUNC, passing AUX as auxiliary data.
+   4. Pass case to PROC_FUNC, passing AUX as auxiliary data.
 
    Returns true if successful, false if an I/O error occurred. */
 bool
@@ -154,10 +148,14 @@
   if (proc_func == NULL
       && case_source_is_class (vfm_source, &storage_source_class)
       && vfm_sink == NULL
-      && !temporary
-      && n_trns == 0)
+      && temporary_trns_chain == NULL
+      && trns_chain_is_empty (permanent_trns_chain))
     {
-      /* Nothing to do. */
+      expr_free (process_if_expr);
+      process_if_expr = NULL;
+      dict_set_case_limit (default_dict, 0);
+      dict_clear_vectors (default_dict);
+
       update_last_vfm_invocation ();
       return true;
     }
@@ -167,8 +165,57 @@
       
       open_active_file ();
       ok = internal_procedure (proc_func, aux);
-      if (!close_active_file ())
-        ok = false;
+      ok = close_active_file () && ok;
+
+      return ok;
+    }
+}
+
+/* Callback function for multipass_procedure(). */
+static bool
+multipass_callback (struct ccase *c, void *cf_) 
+{
+  struct casefile *cf = cf_;
+  return casefile_append (cf, c);
+}
+
+/* Procedure that allows multiple passes over the input data.
+   The entire active file is passed to PROC_FUNC, with the given
+   AUX as auxiliary data, as a unit. */
+bool
+multipass_procedure (bool (*proc_func) (const struct casefile *, void *aux),
+                     void *aux) 
+{
+  if (case_source_is_class (vfm_source, &storage_source_class)
+      && vfm_sink == NULL
+      && temporary_trns_chain == NULL
+      && trns_chain_is_empty (permanent_trns_chain))
+    {
+      proc_func (storage_source_get_casefile (vfm_source), aux);
+
+      expr_free (process_if_expr);
+      process_if_expr = NULL;
+      dict_set_case_limit (default_dict, 0);
+      dict_clear_vectors (default_dict);
+
+      update_last_vfm_invocation ();
+      return true;
+    }
+  else 
+    {
+      struct casefile *cf;
+      bool ok;
+
+      assert (proc_func != NULL);
+
+      cf = casefile_create (dict_get_next_value_idx (default_dict));
+
+      open_active_file ();
+      ok = internal_procedure (multipass_callback, cf);
+      ok = proc_func (cf, aux) && ok;
+      ok = close_active_file () && ok;
+
+      casefile_destroy (cf);
 
       return ok;
     }
@@ -237,25 +284,26 @@
 static void
 open_active_file (void)
 {
-  assert (!in_procedure);
-  in_procedure = true;
-
-  /* Make temp_dict refer to the dictionary right before data
-     reaches the sink */
-  if (!temporary)
-    {
-      temp_trns = n_trns;
-      temp_dict = default_dict;
-    }
+  add_case_limit_trns ();
+  add_filter_trns ();
+  add_process_if_trns ();
+
+  /* Finalize transformations. */
+  trns_chain_finalize (cur_trns_chain);
+
+  /* Make permanent_dict refer to the dictionary right before
+     data reaches the sink. */
+  if (permanent_dict == NULL)
+    permanent_dict = default_dict;
 
   /* Figure out compaction. */
-  compactor = (dict_needs_compaction (temp_dict)
-               ? dict_make_compactor (temp_dict)
+  compactor = (dict_needs_compaction (permanent_dict)
+               ? dict_make_compactor (permanent_dict)
                : NULL);
 
   /* Prepare sink. */
   if (vfm_sink == NULL)
-    vfm_sink = create_case_sink (&storage_sink_class, temp_dict, NULL);
+    vfm_sink = create_case_sink (&storage_sink_class, permanent_dict, NULL);
   if (vfm_sink->class->open != NULL)
     vfm_sink->class->open (vfm_sink);
 
@@ -270,9 +318,6 @@
       for (i = 0; i < n_lag; i++)
         case_nullify (&lag_queue[i]);
     }
-
-  /* Close any unclosed DO IF or LOOP constructs. */
-  ctl_stack_clear ();
 }
 
 /* Transforms trns_case and writes it to the replacement active
@@ -282,25 +327,22 @@
 static bool
 write_case (struct write_case_data *wc_data)
 {
-  int retval;
+  enum trns_result retval;
+  size_t case_nr;
   
   /* Execute permanent transformations.  */
-  retval = execute_transformations (&wc_data->trns_case, t_trns, f_trns,
-                                    temp_trns, wc_data->cases_written + 1);
-  if (retval != 1)
+  case_nr = wc_data->cases_written + 1;
+  retval = trns_chain_execute (permanent_trns_chain,
+                               &wc_data->trns_case, &case_nr);
+  if (retval != TRNS_CONTINUE)
     goto done;
 
-  /* N OF CASES. */
-  if (dict_get_case_limit (default_dict)
-      && wc_data->cases_written >= dict_get_case_limit (default_dict))
-    goto done;
-  wc_data->cases_written++;
-
   /* Write case to LAG queue. */
   if (n_lag)
     lag_case (&wc_data->trns_case);
 
   /* Write case to replacement active file. */
+  wc_data->cases_written++;
   if (vfm_sink->class->write != NULL) 
     {
       if (compactor != NULL) 
@@ -314,94 +356,24 @@
     }
   
   /* Execute temporary transformations. */
-  retval = execute_transformations (&wc_data->trns_case, t_trns, temp_trns,
-                                    n_trns, wc_data->cases_written);
-  if (retval != 1)
-    goto done;
-  
-  /* FILTER, PROCESS IF, post-TEMPORARY N OF CASES. */
-  if (filter_case (&wc_data->trns_case, wc_data->cases_written)
-      || (dict_get_case_limit (temp_dict)
-          && wc_data->cases_analyzed >= dict_get_case_limit (temp_dict)))
-    goto done;
-  wc_data->cases_analyzed++;
+  if (temporary_trns_chain != NULL) 
+    {
+      retval = trns_chain_execute (temporary_trns_chain,
+                                   &wc_data->trns_case,
+                                   &wc_data->cases_written);
+      if (retval != TRNS_CONTINUE)
+        goto done;
+    }
 
   /* Pass case to procedure. */
+  wc_data->cases_analyzed++;
   if (wc_data->proc_func != NULL)
     if (!wc_data->proc_func (&wc_data->trns_case, wc_data->aux))
-      retval = -1;
+      retval = TRNS_ERROR;
 
  done:
   clear_case (&wc_data->trns_case);
-  return retval != -1;
-}
-
-/* Transforms case C using the transformations in TRNS[] with
-   indexes FIRST_IDX through LAST_IDX, exclusive.  Case C will
-   become case CASE_NUM (1-based) in the output file.  Returns 1
-   if the case was successfully transformed, 0 if it was filtered
-   out by one of the transformations, or -1 if the procedure
-   should be abandoned due to a fatal error. */
-static int
-execute_transformations (struct ccase *c,
-                         struct transformation *trns,
-                         int first_idx, int last_idx,
-                         int case_num) 
-{
-  int idx;
-
-  for (idx = first_idx; idx != last_idx; )
-    {
-      struct transformation *t = &trns[idx];
-      int retval = t->proc (t->private, c, case_num);
-      switch (retval)
-        {
-        case TRNS_CONTINUE:
-          idx++;
-          break;
-          
-        case TRNS_DROP_CASE:
-          return 0;
-
-        case TRNS_ERROR:
-          return -1;
-
-        case TRNS_NEXT_CASE:
-          abort ();
-
-        case TRNS_END_FILE:
-          abort ();
-          
-        default:
-          idx = retval;
-          break;
-        }
-    }
-
-  return 1;
-}
-
-/* Returns nonzero if case C with case number CASE_NUM should be
-   excluded as specified on FILTER or PROCESS IF, otherwise
-   zero. */
-static int
-filter_case (const struct ccase *c, int case_idx)
-{
-  /* FILTER. */
-  struct variable *filter_var = dict_get_filter (default_dict);
-  if (filter_var != NULL) 
-    {
-      double f = case_num (c, filter_var->fv);
-      if (f == 0.0 || mv_is_num_missing (&filter_var->miss, f))
-        return 1;
-    }
-
-  /* PROCESS IF. */
-  if (process_if_expr != NULL
-      && expr_evaluate_num (process_if_expr, c, case_idx) != 1.0)
-    return 1;
-
-  return 0;
+  return retval != TRNS_ERROR;
 }
 
 /* Add C to the lag queue. */
@@ -452,13 +424,8 @@
       n_lag = 0;
     }
   
-  /* Dictionary from before TEMPORARY becomes permanent.. */
-  if (temporary)
-    {
-      dict_destroy (default_dict);
-      default_dict = temp_dict;
-      temp_dict = NULL;
-    }
+  /* Dictionary from before TEMPORARY becomes permanent. */
+  proc_cancel_temporary_transformations ();
 
   /* Finish compaction. */
   if (compactor != NULL) 
@@ -479,16 +446,9 @@
 
   /* Cancel TEMPORARY, PROCESS IF, FILTER, N OF CASES, vectors,
      and get rid of all the transformations. */
-  cancel_temporary ();
-  expr_free (process_if_expr);
-  process_if_expr = NULL;
-  dict_set_case_limit (default_dict, 0);
   dict_clear_vectors (default_dict);
-
-  assert (in_procedure);
-  in_procedure = false;
-
-  return cancel_transformations ();
+  permanent_dict = NULL;
+  return proc_cancel_all_transformations ();
 }
 
 /* Returns a pointer to the lagged case from N_BEFORE cases before the
@@ -509,56 +469,6 @@
   else
     return NULL;
 }
-   
-/* Appends TRNS to t_trns[], the list of all transformations to be
-   performed on data as it is read from the active file. */
-void
-add_transformation (trns_proc_func *proc, trns_free_func *free, void *private)
-{
-  struct transformation *trns;
-
-  assert (!in_procedure);
-
-  if (n_trns >= m_trns)
-    t_trns = x2nrealloc (t_trns, &m_trns, sizeof *t_trns);
-  trns = &t_trns[n_trns++];
-  trns->proc = proc;
-  trns->free = free;
-  trns->private = private;
-}
-
-/* Returns the index number that the next transformation added by
-   add_transformation() will receive.  A trns_proc_func that
-   returns this index causes control flow to jump to it. */
-size_t
-next_transformation (void) 
-{
-  return n_trns;
-}
-
-/* Cancels all active transformations, including any transformations
-   created by the input program.
-   Returns true if successful, false if an I/O error occurred. */
-bool
-cancel_transformations (void)
-{
-  bool ok = true;
-  size_t i;
-  for (i = 0; i < n_trns; i++)
-    {
-      struct transformation *t = &t_trns[i];
-      if (t->free != NULL) 
-        {
-          if (!t->free (t->private))
-            ok = false; 
-        }
-    }
-  n_trns = f_trns = 0;
-  free (t_trns);
-  t_trns = NULL;
-  m_trns = 0;
-  return ok;
-}
 
 /* Represents auxiliary data for handling SPLIT FILE. */
 struct split_aux_data 
@@ -714,7 +624,7 @@
   };
 
 static bool multipass_split_callback (struct ccase *c, void *aux_);
-static void multipass_split_output (struct multipass_split_aux_data *);
+static bool multipass_split_output (struct multipass_split_aux_data *);
 
 /* Returns true if successful, false if an I/O error occurred. */
 bool
@@ -736,7 +646,7 @@
 
   ok = internal_procedure (multipass_split_callback, &aux);
   if (aux.casefile != NULL)
-    multipass_split_output (&aux);
+    ok = multipass_split_output (&aux) && ok;
   case_destroy (&aux.prev_case);
 
   if (!close_active_file ())
@@ -750,13 +660,14 @@
 multipass_split_callback (struct ccase *c, void *aux_)
 {
   struct multipass_split_aux_data *aux = aux_;
+  bool ok = true;
 
   /* Start a new series if needed. */
   if (aux->casefile == NULL || !equal_splits (c, &aux->prev_case))
     {
       /* Pass any cases to split_func. */
       if (aux->casefile != NULL)
-        multipass_split_output (aux);
+        ok = multipass_split_output (aux);
 
       /* Start a new casefile. */
       aux->casefile = casefile_create (dict_get_next_value_idx (default_dict));
@@ -767,18 +678,21 @@
       case_clone (&aux->prev_case, c);
     }
 
-  return casefile_append (aux->casefile, c);
+  return casefile_append (aux->casefile, c) && ok;
 }
 
-static void
+static bool
 multipass_split_output (struct multipass_split_aux_data *aux)
 {
+  bool ok;
+  
   assert (aux->casefile != NULL);
-  aux->split_func (aux->casefile, aux->func_aux);
+  ok = aux->split_func (aux->casefile, aux->func_aux);
   casefile_destroy (aux->casefile);
   aux->casefile = NULL;
-}
 
+  return ok;
+}
 
 /* Discards all the current state in preparation for a data-input
    command like DATA LIST or GET. */
@@ -796,12 +710,307 @@
       vfm_source = NULL;
     }
 
-  cancel_transformations ();
-
-  ctl_stack_clear ();
+  proc_cancel_all_transformations ();
 
   expr_free (process_if_expr);
   process_if_expr = NULL;
 
-  cancel_temporary ();
+  proc_cancel_temporary_transformations ();
+}
+
+/* Returns the current set of permanent transformations,
+   and clears the permanent transformations.
+   For use by INPUT PROGRAM. */
+struct trns_chain *
+proc_capture_transformations (void) 
+{
+  struct trns_chain *chain;
+  
+  assert (temporary_trns_chain == NULL);
+  chain = permanent_trns_chain;
+  cur_trns_chain = permanent_trns_chain = trns_chain_create ();
+  return chain;
+}
+
+/* Adds a transformation that processes a case with PROC and
+   frees itself with FREE to the current set of transformations.
+   The functions are passed AUX as auxiliary data. */
+void
+add_transformation (trns_proc_func *proc, trns_free_func *free, void *aux)
+{
+  trns_chain_append (cur_trns_chain, NULL, proc, free, aux);
+}
+
+/* Adds a transformation that processes a case with PROC and
+   frees itself with FREE to the current set of transformations.
+   When parsing of the block of transformations is complete,
+   FINALIZE will be called.
+   The functions are passed AUX as auxiliary data. */
+void
+add_transformation_with_finalizer (trns_finalize_func *finalize,
+                                   trns_proc_func *proc,
+                                   trns_free_func *free, void *aux)
+{
+  trns_chain_append (cur_trns_chain, finalize, proc, free, aux);
+}
+
+/* Returns the index of the next transformation.
+   This value can be returned by a transformation procedure
+   function to indicate a "jump" to that transformation. */
+size_t
+next_transformation (void) 
+{
+  return trns_chain_next (cur_trns_chain);
+}
+
+/* Returns true if the next call to add_transformation() will add
+   a temporary transformation, false if it will add a permanent
+   transformation. */
+bool
+proc_in_temporary_transformations (void) 
+{
+  return temporary_trns_chain != NULL;
+}
+
+/* Marks the start of temporary transformations.
+   Further calls to add_transformation() will add temporary
+   transformations. */
+void
+proc_start_temporary_transformations (void) 
+{
+  if (!proc_in_temporary_transformations ())
+    {
+      add_case_limit_trns ();
+
+      permanent_dict = dict_clone (default_dict);
+      trns_chain_finalize (permanent_trns_chain);
+      temporary_trns_chain = cur_trns_chain = trns_chain_create ();
+    }
+}
+
+/* Converts all the temporary transformations, if any, to
+   permanent transformations.  Further transformations will be
+   permanent.
+   Returns true if anything changed, false otherwise. */
+bool
+proc_make_temporary_transformations_permanent (void) 
+{
+  if (proc_in_temporary_transformations ()) 
+    {
+      trns_chain_finalize (temporary_trns_chain);
+      trns_chain_splice (permanent_trns_chain, temporary_trns_chain);
+      temporary_trns_chain = NULL;
+
+      dict_destroy (permanent_dict);
+      permanent_dict = NULL;
+
+      return true;
+    }
+  else
+    return false;
+}
+
+/* Cancels all temporary transformations, if any.  Further
+   transformations will be permanent.
+   Returns true if anything changed, false otherwise. */
+bool
+proc_cancel_temporary_transformations (void) 
+{
+  if (proc_in_temporary_transformations ()) 
+    {
+      dict_destroy (default_dict);
+      default_dict = permanent_dict;
+      permanent_dict = NULL;
+
+      trns_chain_destroy (temporary_trns_chain);
+      temporary_trns_chain = NULL;
+
+      return true;
+    }
+  else
+    return false;
+}
+
+/* Cancels all transformations, if any.
+   Returns true if successful, false on I/O error. */
+bool
+proc_cancel_all_transformations (void)
+{
+  bool ok;
+  ok = trns_chain_destroy (permanent_trns_chain);
+  ok = trns_chain_destroy (temporary_trns_chain) && ok;
+  permanent_trns_chain = cur_trns_chain = trns_chain_create ();
+  temporary_trns_chain = NULL;
+  return ok;
+}
+
+/* Initializes procedure handling. */
+void
+proc_init (void) 
+{
+  default_dict = dict_create ();
+  proc_cancel_all_transformations ();
+}
+
+/* Finishes up procedure handling. */
+void
+proc_done (void)
+{
+  discard_variables ();
+}
+
+/* Sets SINK as the destination for procedure output from the
+   next procedure. */
+void
+proc_set_sink (struct case_sink *sink) 
+{
+  assert (vfm_sink == NULL);
+  vfm_sink = sink;
+}
+
+/* Sets SOURCE as the source for procedure input for the next
+   procedure. */
+void
+proc_set_source (struct case_source *source) 
+{
+  assert (vfm_source == NULL);
+  vfm_source = source;
+}
+
+/* Returns true if a source for the next procedure has been
+   configured, false otherwise. */
+bool
+proc_has_source (void) 
+{
+  return vfm_source != NULL;
+}
+
+/* Returns the output from the previous procedure.
+   For use only immediately after executing a procedure.
+   The returned casefile is owned by the caller; it will not be
+   automatically used for the next procedure's input. */
+struct casefile *
+proc_capture_output (void) 
+{
+  struct casefile *casefile;
+
+  /* Try to make sure that this function is called immediately
+     after procedure() or a similar function. */
+  assert (vfm_source != NULL);
+  assert (case_source_is_class (vfm_source, &storage_source_class));
+  assert (trns_chain_is_empty (permanent_trns_chain));
+  assert (!proc_in_temporary_transformations ());
+
+  casefile = storage_source_decapsulate (vfm_source);
+  vfm_source = NULL;
+
+  return casefile;
+}
+
+static trns_proc_func case_limit_trns_proc;
+static trns_free_func case_limit_trns_free;
+
+/* Adds a transformation that limits the number of cases that may
+   pass through, if default_dict has a case limit. */
+static void
+add_case_limit_trns (void) 
+{
+  size_t case_limit = dict_get_case_limit (default_dict);
+  if (case_limit != 0)
+    {
+      size_t *cases_remaining = xmalloc (sizeof *cases_remaining);
+      *cases_remaining = case_limit;
+      add_transformation (case_limit_trns_proc, case_limit_trns_free,
+                          cases_remaining);
+      dict_set_case_limit (default_dict, 0);
+    }
+}
+
+/* Limits the maximum number of cases processed to
+   *CASES_REMAINING. */
+static int
+case_limit_trns_proc (void *cases_remaining_,
+                      struct ccase *c UNUSED, int case_nr UNUSED) 
+{
+  size_t *cases_remaining = cases_remaining_;
+  if (*cases_remaining > 0) 
+    {
+      *cases_remaining--;
+      return TRNS_CONTINUE;
+    }
+  else
+    return TRNS_DROP_CASE;
+}
+
+/* Frees the data associated with a case limit transformation. */
+static bool
+case_limit_trns_free (void *cases_remaining_) 
+{
+  size_t *cases_remaining = cases_remaining_;
+  free (cases_remaining);
+  return true;
+}
+
+static trns_proc_func filter_trns_proc;
+
+/* Adds a temporary transformation to filter data according to
+   the variable specified on FILTER, if any. */
+static void
+add_filter_trns (void) 
+{
+  struct variable *filter_var = dict_get_filter (default_dict);
+  if (filter_var != NULL) 
+    {
+      proc_start_temporary_transformations ();
+      add_transformation (filter_trns_proc, NULL, filter_var);
+    }
+}
+
+/* FILTER transformation. */
+static int
+filter_trns_proc (void *filter_var_,
+                  struct ccase *c UNUSED, int case_nr UNUSED) 
+  
+{
+  struct variable *filter_var = filter_var_;
+  double f = case_num (c, filter_var->fv);
+  return (f != 0.0 && !mv_is_num_missing (&filter_var->miss, f)
+          ? TRNS_CONTINUE : TRNS_DROP_CASE);
+}
+
+static trns_proc_func process_if_trns_proc;
+static trns_free_func process_if_trns_free;
+
+/* Adds a temporary transformation to filter data according to
+   the expression specified on PROCESS IF, if any. */
+static void
+add_process_if_trns (void) 
+{
+  if (process_if_expr != NULL) 
+    {
+      proc_start_temporary_transformations ();
+      add_transformation (process_if_trns_proc, process_if_trns_free,
+                          process_if_expr);
+      process_if_expr = NULL;
+    }
+}
+
+/* PROCESS IF transformation. */
+static int
+process_if_trns_proc (void *expression_,
+                      struct ccase *c UNUSED, int case_nr UNUSED) 
+  
+{
+  struct expression *expression = expression_;
+  return (expr_evaluate_num (expression, c, case_nr) == 1.0
+          ? TRNS_CONTINUE : TRNS_DROP_CASE);
+}
+
+/* Frees a PROCESS IF transformation. */
+static bool
+process_if_trns_free (void *expression_) 
+{
+  struct expression *expression = expression_;
+  expr_free (expression);
+  return true;
 }
Index: pspp/src/procedure.h
diff -u pspp/src/procedure.h:1.4 pspp/src/procedure.h:1.5
--- pspp/src/procedure.h:1.4    Thu Apr 27 02:42:01 2006
+++ pspp/src/procedure.h        Thu May  4 06:19:02 2006
@@ -22,26 +22,59 @@
 
 #include <time.h>
 #include <stdbool.h>
+#include <data/transformations.h>
 
 struct ccase;
 struct casefile;
+struct case_sink;
+struct case_source;
 
-/* The current active file, from which cases are read. */
-extern struct case_source *vfm_source;
+/* Dictionary produced by permanent and temporary transformations
+   on data from the source. */
+extern struct dictionary *default_dict;
+
+/* Transformations. */
+
+void add_transformation (trns_proc_func *, trns_free_func *, void *);
+void add_transformation_with_finalizer (trns_finalize_func *,
+                                        trns_proc_func *,
+                                        trns_free_func *, void *);
+size_t next_transformation (void);
+
+void discard_variables (void);
+
+bool proc_cancel_all_transformations (void);
+struct trns_chain *proc_capture_transformations (void);
+
+void proc_start_temporary_transformations (void);
+bool proc_in_temporary_transformations (void);
+bool proc_make_temporary_transformations_permanent (void);
+bool proc_cancel_temporary_transformations (void);
+
+/* Procedures. */
 
-/* The replacement active file, to which cases are written. */
-extern struct case_sink *vfm_sink;
+void proc_init (void);
+void proc_done (void);
+
+void proc_set_source (struct case_source *);
+bool proc_has_source (void);
+
+void proc_set_sink (struct case_sink *);
+struct casefile *proc_capture_output (void);
 
 bool procedure (bool (*proc_func) (struct ccase *, void *aux), void *aux);
 bool procedure_with_splits (void (*begin_func) (void *aux),
                             bool (*proc_func) (struct ccase *, void *aux),
                             void (*end_func) (void *aux),
                             void *aux);
+bool multipass_procedure (bool (*proc_func) (const struct casefile *,
+                                             void *aux),
+                          void *aux);
 bool multipass_procedure_with_splits (bool (*) (const struct casefile *,
                                                 void *),
                                       void *aux);
 time_t time_of_last_procedure (void);
-
+
 /* Number of cases to lag. */
 extern int n_lag;
 
Index: pspp/src/ui/terminal/ChangeLog
diff -u pspp/src/ui/terminal/ChangeLog:1.8 pspp/src/ui/terminal/ChangeLog:1.9
--- pspp/src/ui/terminal/ChangeLog:1.8  Wed Apr 26 20:54:08 2006
+++ pspp/src/ui/terminal/ChangeLog      Thu May  4 06:19:02 2006
@@ -1,3 +1,14 @@
+Wed May  3 23:09:37 2006  Ben Pfaff  <address@hidden>
+
+       Continue reforming procedure execution.  In this phase, get rid of
+       many global variables, consolidating procedure execution in
+       procedure.c.  Encapsulate transformations in new "struct
+       trns_chain".  Also, change implementation of N OF CASES, FILTER,
+       and PROCESS IF from special cases to transformations.
+        
+       * main.c: (main) Use proc_init().
+       (terminate) Use proc_done().
+
 Wed Apr 26 13:34:54 2006  Ben Pfaff  <address@hidden>
 
        Improve command name completion in readline.
Index: pspp/src/ui/terminal/main.c
diff -u pspp/src/ui/terminal/main.c:1.10 pspp/src/ui/terminal/main.c:1.11
--- pspp/src/ui/terminal/main.c:1.10    Tue May  2 01:29:06 2006
+++ pspp/src/ui/terminal/main.c Thu May  4 06:19:02 2006
@@ -19,6 +19,9 @@
 
 #include <config.h>
 
+#include <signal.h>
+#include <stdio.h>
+
 #include "command-line.h"
 #include "msg-ui.h"
 #include "progname.h"
@@ -39,9 +42,7 @@
 #include <libpspp/version.h>
 #include <math/random.h>
 #include <output/output.h>
-#include <signal.h>
-#include <stdio.h>
-
+#include <procedure.h>
 
 #if HAVE_FPU_CONTROL_H
 #include <fpu_control.h>
@@ -93,8 +94,7 @@
   readln_initialize ();
   settings_init ();
   random_init ();
-
-  default_dict = dict_create ();
+  proc_init ();
 
   if (parse_command_line (argc, argv)) 
     {
@@ -135,7 +135,8 @@
      Any lines read after the first token must be continuation
      lines. */
   getl_set_prompt_style (GETL_PROMPT_LATER);
-  return cmd_parse (vfm_source != NULL ? CMD_STATE_DATA : CMD_STATE_INITIAL);
+  return cmd_parse (proc_has_source ()
+                    ? CMD_STATE_DATA : CMD_STATE_INITIAL);
 }
 
 static void
@@ -197,8 +198,7 @@
     {
       terminating = true;
 
-      cancel_transformations ();
-      dict_destroy (default_dict);
+      proc_done ();
 
       random_done ();
       settings_done ();




reply via email to

[Prev in Thread] Current Thread [Next in Thread]