gawk-diffs
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[SCM] gawk branch, feature/readall, updated. gawk-4.1.0-4596-g508e936


From: Andrew J. Schorr
Subject: [SCM] gawk branch, feature/readall, updated. gawk-4.1.0-4596-g508e936
Date: Wed, 8 Dec 2021 17:07:00 -0500 (EST)

This is an automated email from the git hooks/post-receive script. It was
generated because a ref change was pushed to the repository containing
the project "gawk".

The branch, feature/readall has been updated
       via  508e9368d2d461b2290af6787cad6dfbed357176 (commit)
      from  c4b80d4a006d3cbcce3aaca1c598a094de936f8b (commit)

Those revisions listed above that are new to this repository have
not appeared on any other notification email; so we list those
revisions in full, below.

- Log -----------------------------------------------------------------
http://git.sv.gnu.org/cgit/gawk.git/commit/?id=508e9368d2d461b2290af6787cad6dfbed357176

commit 508e9368d2d461b2290af6787cad6dfbed357176
Author: Andrew J. Schorr <aschorr@telemetry-investments.com>
Date:   Wed Dec 8 17:05:52 2021 -0500

    Enhance rwarray extension to add writeall and readall functions for dumping 
and reloading global state.

diff --git a/doc/ChangeLog b/doc/ChangeLog
index 6625109..6c5439b 100644
--- a/doc/ChangeLog
+++ b/doc/ChangeLog
@@ -1,3 +1,8 @@
+2021-12-08         Andrew J. Schorr      <aschorr@telemetry-investments.com>
+
+       * gawktexi.in: Document new rwarray functions writeall and readall.
+       * wordlist: Add readall and writeall.
+
 2021-11-25         Arnold D. Robbins     <arnold@skeeve.com>
 
        * gawktexi.in: Add missing @item for AWKgo. Thanks to Antonio
diff --git a/doc/gawk.info b/doc/gawk.info
index 4f76825..6aaff40 100644
--- a/doc/gawk.info
+++ b/doc/gawk.info
@@ -29128,8 +29128,8 @@ File: gawk.info,  Node: Extension Sample Read write 
array,  Next: Extension Samp
 17.7.9 Dumping and Restoring an Array
 -------------------------------------
 
-The 'rwarray' extension adds two functions, named 'writea()' and
-'reada()', as follows:
+The 'rwarray' extension adds four functions, named 'writea()',
+'reada()', 'writeall()' and 'readall()', as follows:
 
 '@load "rwarray"'
      This is how you load the extension.
@@ -29146,6 +29146,20 @@ The 'rwarray' extension adds two functions, named 
'writea()' and
      argument.  It clears the array first.  Here too, the return value
      is one on success, or zero upon failure.
 
+'ret = writeall(file)'
+     This function takes a string argument, which is the name of the
+     file to which to dump the state of all variables.  Calling this
+     function is completely equivalent to calling 'writea(file,
+     SYMTAB)'.  It returns one on success, or zero upon failure
+
+'ret = writeall(file)'
+     This function takes a string argument, which is the name of the
+     file from which to read the contents of various global variables.
+     For each variable in the file, the data is loaded unless the
+     variable already exists.  If the variable already exists, the data
+     for that variable in the file is ignored.  It returns one on
+     success, or zero upon failure.
+
    The array created by 'reada()' is identical to that written by
 'writea()' in the sense that the contents are the same.  However, due to
 implementation issues, the array traversal order of the re-created array
@@ -29161,6 +29175,13 @@ written as native binary data.  Thus, arrays 
containing only string data
 can theoretically be dumped on systems with one byte order and restored
 on systems with a different one, but this has not been tried.
 
+   Note that the 'writeall()' and 'readall()' functions provide a
+mechanism for maintaining persistent state across repeated invocations
+of a program.  If, for example, a program calculates some statistics
+based on the data in a series of files, it could save state using
+'writeall()' after processing N files, and then reload the state using
+'readall()' when the N+1st file arrives to update the results.
+
    Here is an example:
 
      @load "rwarray"
@@ -29168,6 +29189,10 @@ on systems with a different one, but this has not been 
tried.
      ret = writea("arraydump.bin", array)
      ...
      ret = reada("arraydump.bin", array)
+     ...
+     ret = writeall("globalstate.bin")
+     ...
+     ret = readall("globalstate.bin")
 
 
 File: gawk.info,  Node: Extension Sample Readfile,  Next: Extension Sample 
Time,  Prev: Extension Sample Read write array,  Up: Extension Samples
@@ -37998,6 +38023,8 @@ Index
                                                               (line  18)
 * readable data files, checking:         File Checking.       (line   6)
 * readable.awk program:                  File Checking.       (line  11)
+* readall() extension function:          Extension Sample Read write array.
+                                                              (line  30)
 * readdir extension:                     Extension Sample Readdir.
                                                               (line   9)
 * readfile() extension function:         Extension Sample Readfile.
@@ -38711,6 +38738,8 @@ Index
 * words, usage counts, generating:       Word Sorting.        (line   6)
 * writea() extension function:           Extension Sample Read write array.
                                                               (line  12)
+* writeall() extension function:         Extension Sample Read write array.
+                                                              (line  24)
 * xgettext utility:                      String Extraction.   (line  13)
 * xor:                                   Bitwise Functions.   (line  58)
 * XOR bitwise operation:                 Bitwise Functions.   (line   6)
@@ -39246,92 +39275,92 @@ Ref: table-readdir-file-types1174550
 Node: Extension Sample Revout1175618
 Node: Extension Sample Rev2way1176207
 Node: Extension Sample Read write array1176947
-Node: Extension Sample Readfile1178889
-Node: Extension Sample Time1179984
-Node: Extension Sample API Tests1181736
-Node: gawkextlib1182228
-Node: Extension summary1185146
-Node: Extension Exercises1188848
-Node: Language History1190090
-Node: V7/SVR3.11191746
-Node: SVR41193898
-Node: POSIX1195332
-Node: BTL1196713
-Node: POSIX/GNU1197442
-Node: Feature History1203220
-Node: Common Extensions1220395
-Node: Ranges and Locales1221678
-Ref: Ranges and Locales-Footnote-11226294
-Ref: Ranges and Locales-Footnote-21226321
-Ref: Ranges and Locales-Footnote-31226556
-Node: Contributors1226779
-Node: History summary1232776
-Node: Installation1234156
-Node: Gawk Distribution1235100
-Node: Getting1235584
-Node: Extracting1236547
-Node: Distribution contents1238185
-Node: Unix Installation1245246
-Node: Quick Installation1246050
-Node: Compiling with MPFR1248470
-Node: Shell Startup Files1249160
-Node: Additional Configuration Options1250249
-Node: Configuration Philosophy1252564
-Node: Compiling from Git1254960
-Node: Building the Documentation1255515
-Node: Non-Unix Installation1256899
-Node: PC Installation1257359
-Node: PC Binary Installation1258197
-Node: PC Compiling1259070
-Node: PC Using1260187
-Node: Cygwin1263740
-Node: MSYS1264964
-Node: VMS Installation1265566
-Node: VMS Compilation1266285
-Ref: VMS Compilation-Footnote-11267514
-Node: VMS Dynamic Extensions1267572
-Node: VMS Installation Details1269257
-Node: VMS Running1271519
-Node: VMS GNV1275798
-Node: Bugs1276512
-Node: Bug definition1277424
-Node: Bug address1280360
-Node: Usenet1283748
-Node: Performance bugs1284937
-Node: Asking for help1287858
-Node: Maintainers1289825
-Node: Other Versions1291019
-Node: Installation summary1299183
-Node: Notes1300547
-Node: Compatibility Mode1301341
-Node: Additions1302123
-Node: Accessing The Source1303048
-Node: Adding Code1304485
-Node: New Ports1310677
-Node: Derived Files1315052
-Ref: Derived Files-Footnote-11320712
-Ref: Derived Files-Footnote-21320747
-Ref: Derived Files-Footnote-31321345
-Node: Future Extensions1321459
-Node: Implementation Limitations1322117
-Node: Extension Design1323327
-Node: Old Extension Problems1324471
-Ref: Old Extension Problems-Footnote-11325989
-Node: Extension New Mechanism Goals1326046
-Ref: Extension New Mechanism Goals-Footnote-11329410
-Node: Extension Other Design Decisions1329599
-Node: Extension Future Growth1331712
-Node: Notes summary1332318
-Node: Basic Concepts1333476
-Node: Basic High Level1334157
-Ref: figure-general-flow1334439
-Ref: figure-process-flow1335125
-Ref: Basic High Level-Footnote-11338427
-Node: Basic Data Typing1338612
-Node: Glossary1341940
-Node: Copying1373827
-Node: GNU Free Documentation License1411370
-Node: Index1436490
+Node: Extension Sample Readfile1180113
+Node: Extension Sample Time1181208
+Node: Extension Sample API Tests1182960
+Node: gawkextlib1183452
+Node: Extension summary1186370
+Node: Extension Exercises1190072
+Node: Language History1191314
+Node: V7/SVR3.11192970
+Node: SVR41195122
+Node: POSIX1196556
+Node: BTL1197937
+Node: POSIX/GNU1198666
+Node: Feature History1204444
+Node: Common Extensions1221619
+Node: Ranges and Locales1222902
+Ref: Ranges and Locales-Footnote-11227518
+Ref: Ranges and Locales-Footnote-21227545
+Ref: Ranges and Locales-Footnote-31227780
+Node: Contributors1228003
+Node: History summary1234000
+Node: Installation1235380
+Node: Gawk Distribution1236324
+Node: Getting1236808
+Node: Extracting1237771
+Node: Distribution contents1239409
+Node: Unix Installation1246470
+Node: Quick Installation1247274
+Node: Compiling with MPFR1249694
+Node: Shell Startup Files1250384
+Node: Additional Configuration Options1251473
+Node: Configuration Philosophy1253788
+Node: Compiling from Git1256184
+Node: Building the Documentation1256739
+Node: Non-Unix Installation1258123
+Node: PC Installation1258583
+Node: PC Binary Installation1259421
+Node: PC Compiling1260294
+Node: PC Using1261411
+Node: Cygwin1264964
+Node: MSYS1266188
+Node: VMS Installation1266790
+Node: VMS Compilation1267509
+Ref: VMS Compilation-Footnote-11268738
+Node: VMS Dynamic Extensions1268796
+Node: VMS Installation Details1270481
+Node: VMS Running1272743
+Node: VMS GNV1277022
+Node: Bugs1277736
+Node: Bug definition1278648
+Node: Bug address1281584
+Node: Usenet1284972
+Node: Performance bugs1286161
+Node: Asking for help1289082
+Node: Maintainers1291049
+Node: Other Versions1292243
+Node: Installation summary1300407
+Node: Notes1301771
+Node: Compatibility Mode1302565
+Node: Additions1303347
+Node: Accessing The Source1304272
+Node: Adding Code1305709
+Node: New Ports1311901
+Node: Derived Files1316276
+Ref: Derived Files-Footnote-11321936
+Ref: Derived Files-Footnote-21321971
+Ref: Derived Files-Footnote-31322569
+Node: Future Extensions1322683
+Node: Implementation Limitations1323341
+Node: Extension Design1324551
+Node: Old Extension Problems1325695
+Ref: Old Extension Problems-Footnote-11327213
+Node: Extension New Mechanism Goals1327270
+Ref: Extension New Mechanism Goals-Footnote-11330634
+Node: Extension Other Design Decisions1330823
+Node: Extension Future Growth1332936
+Node: Notes summary1333542
+Node: Basic Concepts1334700
+Node: Basic High Level1335381
+Ref: figure-general-flow1335663
+Ref: figure-process-flow1336349
+Ref: Basic High Level-Footnote-11339651
+Node: Basic Data Typing1339836
+Node: Glossary1343164
+Node: Copying1375051
+Node: GNU Free Documentation License1412594
+Node: Index1437714
 
 End Tag Table
 
diff --git a/doc/gawk.texi b/doc/gawk.texi
index d476b74..cb7ad4c 100644
--- a/doc/gawk.texi
+++ b/doc/gawk.texi
@@ -39729,8 +39729,9 @@ is:
 @node Extension Sample Read write array
 @subsection Dumping and Restoring an Array
 
-The @code{rwarray} extension adds two functions,
-named @code{writea()} and @code{reada()}, as follows:
+The @code{rwarray} extension adds four functions,
+named @code{writea()}, @code{reada()},
+@code{writeall()} and @code{readall()}, as follows:
 
 @table @code
 @item @@load "rwarray"
@@ -39749,6 +39750,24 @@ success, or zero upon failure.
 it reads the file named as its first argument, filling in
 the array named as the second argument. It clears the array first.
 Here too, the return value is one on success, or zero upon failure.
+
+@cindex @code{writeall()} extension function
+@item ret = writeall(file)
+This function takes a string argument, which is the name of the file
+to which to dump the state of all variables.
+Calling this function
+is completely equivalent to calling
+@code{writea(file, SYMTAB)}.
+It returns one on success, or zero upon failure
+
+@cindex @code{readall()} extension function
+@item ret = writeall(file)
+This function takes a string argument, which is the name of the
+file from which to read the contents of various global variables.
+For each variable in the file, the data is loaded unless the variable
+already exists. If the variable already exists, the data for that variable
+in the file is ignored.
+It returns one on success, or zero upon failure.
 @end table
 
 The array created by @code{reada()} is identical to that written by
@@ -39766,6 +39785,13 @@ as native binary data.  Thus, arrays containing only 
string data can
 theoretically be dumped on systems with one byte order and restored on
 systems with a different one, but this has not been tried.
 
+Note that the @code{writeall()} and @code{readall()} functions provide
+a mechanism for maintaining persistent state across repeated invocations of a
+program. If, for example, a program calculates some statistics based on the
+data in a series of files, it could save state using @code{writeall()} after
+processing N files, and then reload the state using @code{readall()} when
+the N+1st file arrives to update the results.
+
 Here is an example:
 
 @example
@@ -39774,6 +39800,10 @@ Here is an example:
 ret = writea("arraydump.bin", array)
 @dots{}
 ret = reada("arraydump.bin", array)
+@dots{}
+ret = writeall("globalstate.bin")
+@dots{}
+ret = readall("globalstate.bin")
 @end example
 
 @node Extension Sample Readfile
diff --git a/doc/gawktexi.in b/doc/gawktexi.in
index f3720f1..97d21f0 100644
--- a/doc/gawktexi.in
+++ b/doc/gawktexi.in
@@ -38572,8 +38572,9 @@ is:
 @node Extension Sample Read write array
 @subsection Dumping and Restoring an Array
 
-The @code{rwarray} extension adds two functions,
-named @code{writea()} and @code{reada()}, as follows:
+The @code{rwarray} extension adds four functions,
+named @code{writea()}, @code{reada()},
+@code{writeall()} and @code{readall()}, as follows:
 
 @table @code
 @item @@load "rwarray"
@@ -38592,6 +38593,24 @@ success, or zero upon failure.
 it reads the file named as its first argument, filling in
 the array named as the second argument. It clears the array first.
 Here too, the return value is one on success, or zero upon failure.
+
+@cindex @code{writeall()} extension function
+@item ret = writeall(file)
+This function takes a string argument, which is the name of the file
+to which to dump the state of all variables.
+Calling this function
+is completely equivalent to calling
+@code{writea(file, SYMTAB)}.
+It returns one on success, or zero upon failure
+
+@cindex @code{readall()} extension function
+@item ret = writeall(file)
+This function takes a string argument, which is the name of the
+file from which to read the contents of various global variables.
+For each variable in the file, the data is loaded unless the variable
+already exists. If the variable already exists, the data for that variable
+in the file is ignored.
+It returns one on success, or zero upon failure.
 @end table
 
 The array created by @code{reada()} is identical to that written by
@@ -38609,6 +38628,13 @@ as native binary data.  Thus, arrays containing only 
string data can
 theoretically be dumped on systems with one byte order and restored on
 systems with a different one, but this has not been tried.
 
+Note that the @code{writeall()} and @code{readall()} functions provide
+a mechanism for maintaining persistent state across repeated invocations of a
+program. If, for example, a program calculates some statistics based on the
+data in a series of files, it could save state using @code{writeall()} after
+processing N files, and then reload the state using @code{readall()} when
+the N+1st file arrives to update the results.
+
 Here is an example:
 
 @example
@@ -38617,6 +38643,10 @@ Here is an example:
 ret = writea("arraydump.bin", array)
 @dots{}
 ret = reada("arraydump.bin", array)
+@dots{}
+ret = writeall("globalstate.bin")
+@dots{}
+ret = readall("globalstate.bin")
 @end example
 
 @node Extension Sample Readfile
diff --git a/doc/wordlist b/doc/wordlist
index 1ad1c42..fd7c181 100644
--- a/doc/wordlist
+++ b/doc/wordlist
@@ -1510,6 +1510,7 @@ rapidjson
 rdev
 rdquo
 reada
+readall
 readdir
 readfile
 readline
@@ -1796,6 +1797,7 @@ wnewmail
 wordfreq
 wr
 writea
+writeall
 www
 wy
 xA
diff --git a/extension/ChangeLog b/extension/ChangeLog
index 0339153..1a0e34a 100644
--- a/extension/ChangeLog
+++ b/extension/ChangeLog
@@ -1,5 +1,43 @@
 2021-12-08         Andrew J. Schorr      <aschorr@telemetry-investments.com>
 
+       * rwarray.c: Add new functions writeall and readall to implement
+       persistent state.
+       (write_backend): New helper function containing most of the logic
+       from do_writea. Note that we do not need to check nargs < 2 because
+       gawk will issue a fatal error if a function is called with fewer
+       than min_required_args. Clean up some minor issues with error
+       handling.
+       (do_writea): Grab the array argument and use write_backend to
+       do the rest of the work.
+       (do_writeall): Lookup SYMTAB and invoke write_backend.
+       (free_value): New function to free memory for data we end up ignoring
+       because the variables exist already.
+       (do_poke): Attempt to create variables that don't exist already or
+       are undefined.
+       (regular_array_handle): Wrapper around create_array.
+       (global_array_handle): Call create_array unless the variable exists
+       already and is an array with zero elements.
+       (read_global): New function used by readall to load global variables
+       from a file.
+       (read_one): New function to read a single array from a file.
+       (read_backend): New helper function containing most of the logic
+       from do_reada. Remove the superfluous nargs check. Read the file
+       prologue and then call read_global or read_one as appropriate to load
+       the data.
+       (do_reada): Grab the array argument and call read_backend with
+       read_one to load the data.
+       (do_readall): Call read_backend with read_global to load the data.
+       (read_array): Call read_elem with additional arg regular_array_handle.
+       (read_elem): Add a function argument controlling array creation to
+       pass down to read_value.
+       (read_value): Add a function argument to call for array creation
+       instead of calling create_array directly, since we may need to use
+       an existing array when populating global arrays in readall.
+       (func_table): Add writeall and readall.
+       * rwarray.3am: Document new functions writeall and readall.
+
+2021-12-08         Andrew J. Schorr      <aschorr@telemetry-investments.com>
+
        * testext.c (test_array_create): New function to create an array
        by name that enables testing whether an undefined variable can
        be converted by the API into an array.
diff --git a/extension/rwarray.3am b/extension/rwarray.3am
index f17ffaa..b10545a 100644
--- a/extension/rwarray.3am
+++ b/extension/rwarray.3am
@@ -1,6 +1,6 @@
 .TH RWARRAY 3am "Feb 02 2018" "Free Software Foundation" "GNU Awk Extension 
Modules"
 .SH NAME
-writea, reada \- write and read gawk arrays to/from files
+writea, reada, writeall, readall \- write and read gawk arrays to/from files
 .SH SYNOPSIS
 .ft CW
 @load "rwarray"
@@ -8,14 +8,20 @@ writea, reada \- write and read gawk arrays to/from files
 ret = writea(file, array)
 .br
 ret = reada(file, array)
+.br
+ret = writeall(file)
+.br
+ret = readall(file)
 .ft R
 .SH DESCRIPTION
 The
 .I rwarray
-extension adds two functions named
-.BR writea() .
-and
+extension adds functions named
+.BR writea() ,
 .BR reada() ,
+.BR writeaall() ,
+and
+.BR readaall() ,
 as follows.
 .TP
 .B writea()
@@ -33,6 +39,23 @@ it reads the file named as its first argument, filling in
 the array named as the second argument. It clears the array
 first.
 Here too, the return value is one on success and zero upon failure.
+.TP
+.B writeall()
+This function takes a string argument, which is the name of the
+file to which dump the state of all variables. Calling this function
+is completely equivalent to calling
+.B writea()
+with the second argument equal to
+.BR SYMTAB .
+It returns one on success, or zero upon failure.
+.TP
+.B readall()
+This function takes a string argument, which is the name of the
+file from which to read the contents of various global variables.
+For each variable in the file, the data is loaded unless the variable
+already exists. If the variable already exists, the data for that variable
+in the file is ignored.
+It returns one on success, or zero upon failure.
 .SH NOTES
 The array created by
 .B reada()
@@ -62,6 +85,10 @@ restored on systems with a different one, but this has not 
been tried.
 ret = writea("arraydump.bin", array)
 \&...
 ret = reada("arraydump.bin", array)
+\&...
+ret = writeall("globalstate.bin")
+\&...
+ret = readall("globalstate.bin")
 .fi
 .ft R
 .SH "SEE ALSO"
diff --git a/extension/rwarray.c b/extension/rwarray.c
index ecc0c4a..4dbfaa8 100644
--- a/extension/rwarray.c
+++ b/extension/rwarray.c
@@ -78,9 +78,10 @@ static awk_bool_t write_elem(FILE *fp, awk_element_t 
*element);
 static awk_bool_t write_value(FILE *fp, awk_value_t *val);
 static awk_bool_t write_number(FILE *fp, awk_value_t *val);
 
+typedef awk_array_t (*array_handle_t)(awk_value_t *);
 static awk_bool_t read_array(FILE *fp, awk_array_t array);
-static awk_bool_t read_elem(FILE *fp, awk_element_t *element);
-static awk_bool_t read_value(FILE *fp, awk_value_t *value);
+static awk_bool_t read_elem(FILE *fp, awk_element_t *element, array_handle_t);
+static awk_bool_t read_value(FILE *fp, awk_value_t *value, array_handle_t, 
awk_value_t *idx);
 static awk_bool_t read_number(FILE *fp, awk_value_t *value, uint32_t code);
 
 /*
@@ -117,12 +118,12 @@ static awk_bool_t read_number(FILE *fp, awk_value_t 
*value, uint32_t code);
 #define VT_BOOL                8
 #define VT_UNDEFINED   20
 
-/* do_writea --- write an array */
+/* write_backend --- write an array */
 
 static awk_value_t *
-do_writea(int nargs, awk_value_t *result, struct awk_ext_func *unused)
+write_backend(awk_value_t *result, awk_array_t array, const char *name)
 {
-       awk_value_t filename, array;
+       awk_value_t filename;
        FILE *fp = NULL;
        uint32_t major = MAJOR;
        uint32_t minor = MINOR;
@@ -130,18 +131,9 @@ do_writea(int nargs, awk_value_t *result, struct 
awk_ext_func *unused)
        assert(result != NULL);
        make_number(0.0, result);
 
-       if (nargs < 2)
-               goto out;
-
-       /* filename is first arg, array to dump is second */
+       /* filename is first arg */
        if (! get_argument(0, AWK_STRING, & filename)) {
-               warning(ext_id, _("do_writea: first argument is not a string"));
-               errno = EINVAL;
-               goto done1;
-       }
-
-       if (! get_argument(1, AWK_ARRAY, & array)) {
-               warning(ext_id, _("do_writea: second argument is not an 
array"));
+               warning(ext_id, _("%s: first argument is not a string"), name);
                errno = EINVAL;
                goto done1;
        }
@@ -162,21 +154,55 @@ do_writea(int nargs, awk_value_t *result, struct 
awk_ext_func *unused)
        if (fwrite(& minor, 1, sizeof(minor), fp) != sizeof(minor))
                goto done1;
 
-       if (write_array(fp, array.array_cookie)) {
+       if (write_array(fp, array)) {
                make_number(1.0, result);
-               goto done0;
+               fclose(fp);
+               return result;
        }
 
 done1:
        update_ERRNO_int(errno);
-       unlink(filename.str_value.str);
-
-done0:
-       fclose(fp);
-out:
+       if (fp != NULL) {
+               fclose(fp);
+               unlink(filename.str_value.str);
+       }
        return result;
 }
 
+/* do_writea --- write an array */
+
+static awk_value_t *
+do_writea(int nargs, awk_value_t *result, struct awk_ext_func *unused)
+{
+       awk_value_t array;
+
+       if (! get_argument(1, AWK_ARRAY, & array)) {
+               warning(ext_id, _("writea: second argument is not an array"));
+               errno = EINVAL;
+               update_ERRNO_int(errno);
+               make_number(0.0, result);
+               return result;
+       }
+       return write_backend(result, array.array_cookie, "writea");
+}
+
+/* do_writeall --- write out SYMTAB */
+
+static awk_value_t *
+do_writeall(int nargs, awk_value_t *result, struct awk_ext_func *unused)
+{
+       awk_value_t array;
+
+       if (! sym_lookup("SYMTAB", AWK_ARRAY, & array)) {
+               warning(ext_id, _("writeall: unable to find SYMTAB array"));
+               errno = EINVAL;
+               update_ERRNO_int(errno);
+               make_number(0.0, result);
+               return result;
+       }
+       return write_backend(result, array.array_cookie, "writeall");
+}
+
 
 /* write_array --- write out an array or a sub-array */
 
@@ -363,12 +389,138 @@ write_number(FILE *fp, awk_value_t *val)
        return awk_true;
 }
 
-/* do_reada --- read an array */
+/* free_value --- release memory for ignored global variables */
+
+static void
+free_value(awk_value_t *v)
+{
+       switch (v->val_type) {
+       case AWK_ARRAY:
+               clear_array(v->array_cookie);
+               break;
+       case AWK_STRING:
+       case AWK_REGEX:
+       case AWK_STRNUM:
+       case AWK_UNDEFINED:
+               gawk_free(v->str_value.str);
+               break;
+       case AWK_BOOL:
+               /* no memory allocated */
+               break;
+       case AWK_NUMBER:
+               switch (v->num_type) {
+               case AWK_NUMBER_TYPE_DOUBLE:
+                       /* no memory allocated */
+                       break;
+               case AWK_NUMBER_TYPE_MPZ:
+                       mpz_clear(v->num_ptr);
+                       break;
+               case AWK_NUMBER_TYPE_MPFR:
+                       mpfr_clear(v->num_ptr);
+                       break;
+               default:
+                       warning(ext_id, _("cannot free number with unknown type 
%d"), v->num_type);
+                       break;
+               }
+               break;
+       default:
+               warning(ext_id, _("cannot free value with unhandled type %d"), 
v->val_type);
+               break;
+       }
+}
+
+/* do_poke --- create a global variable */
+
+static awk_bool_t
+do_poke(awk_element_t *e)
+{
+       awk_value_t t;
+
+       if (e->index.val_type != AWK_STRING)
+               return awk_false;
+       /* So this is a bit tricky. If the program refers to the variable,
+        * then it will already exist in an undefined state after parsing.
+        * If the program never refers to it, then the lookup fails.
+        * We still need to create it in case the program accesses it via
+        * indirection through the SYMTAB table. */
+       if (sym_lookup(e->index.str_value.str, AWK_UNDEFINED, &t) && 
(t.val_type != AWK_UNDEFINED))
+               return awk_false;
+       if (! sym_update(e->index.str_value.str, & e->value)) {
+               warning(ext_id, _("readall: unable to set %s"), 
e->index.str_value.str);
+               return awk_false;
+       }
+       return awk_true;
+}
+
+/* regular_array_handle --- array creation hook for normal reada */
+
+static awk_array_t
+regular_array_handle(awk_value_t *unused)
+{
+       return create_array();
+}
+
+/* global_array_handle --- array creation hook for readall */
+
+static awk_array_t
+global_array_handle(awk_value_t *n)
+{
+       awk_value_t t;
+       size_t count;
+
+       /* The array may exist already because it was instantiated during
+        * program parsing, so we use the existing array if it is empty. */
+       return ((n->val_type == AWK_STRING) && sym_lookup(n->str_value.str, 
AWK_UNDEFINED, &t) && (t.val_type == AWK_ARRAY) && 
get_element_count(t.array_cookie, & count) && ! count) ? t.array_cookie : 
create_array();
+}
+
+/* read_global --- read top-level variables dumped from SYMTAB */
+
+static awk_bool_t
+read_global(FILE *fp, awk_array_t unused)
+{
+       uint32_t i;
+       uint32_t count;
+       awk_element_t new_elem;
+
+       if (fread(& count, 1, sizeof(count), fp) != sizeof(count))
+               return awk_false;
+
+       count = ntohl(count);
+
+       for (i = 0; i < count; i++) {
+               if (read_elem(fp, & new_elem, global_array_handle)) {
+                       if (! do_poke(& new_elem))
+                               free_value(& new_elem.value);
+                       if (new_elem.index.str_value.len)
+                               /* free string allocated by make_const_string */
+                               gawk_free(new_elem.index.str_value.str);
+               } else
+                       return awk_false;
+       }
+
+       return awk_true;
+}
+
+/* read_one --- read one array */
+
+static awk_bool_t
+read_one(FILE *fp, awk_array_t array)
+{
+       if (! clear_array(array)) {
+               errno = ENOMEM;
+               warning(ext_id, _("reada: clear_array failed"));
+               return awk_false;
+       }
+
+       return read_array(fp, array);
+}
+
+/* read_backend --- common code for reada and readall */
 
 static awk_value_t *
-do_reada(int nargs, awk_value_t *result, struct awk_ext_func *unused)
+read_backend(awk_value_t *result, awk_array_t array, const char *name, 
awk_bool_t (*func)(FILE *, awk_array_t))
 {
-       awk_value_t filename, array;
+       awk_value_t filename;
        FILE *fp = NULL;
        uint32_t major;
        uint32_t minor;
@@ -377,18 +529,9 @@ do_reada(int nargs, awk_value_t *result, struct 
awk_ext_func *unused)
        assert(result != NULL);
        make_number(0.0, result);
 
-       if (nargs < 2)
-               goto out;
-
-       /* directory is first arg, array to read is second */
+       /* filename is first arg */
        if (! get_argument(0, AWK_STRING, & filename)) {
-               warning(ext_id, _("do_reada: first argument is not a string"));
-               errno = EINVAL;
-               goto done1;
-       }
-
-       if (! get_argument(1, AWK_ARRAY, & array)) {
-               warning(ext_id, _("do_reada: second argument is not an array"));
+               warning(ext_id, _("%s: first argument is not a string"), name);
                errno = EINVAL;
                goto done1;
        }
@@ -430,13 +573,7 @@ do_reada(int nargs, awk_value_t *result, struct 
awk_ext_func *unused)
                goto done1;
        }
 
-       if (! clear_array(array.array_cookie)) {
-               errno = ENOMEM;
-               warning(ext_id, _("do_reada: clear_array failed"));
-               goto done1;
-       }
-
-       if (read_array(fp, array.array_cookie)) {
+       if ((*func)(fp, array)) {
                make_number(1.0, result);
                goto done0;
        }
@@ -446,10 +583,34 @@ done1:
 done0:
        if (fp != NULL)
                fclose(fp);
-out:
        return result;
 }
 
+/* do_reada --- read an array */
+
+static awk_value_t *
+do_reada(int nargs, awk_value_t *result, struct awk_ext_func *unused)
+{
+       awk_value_t array;
+
+       if (! get_argument(1, AWK_ARRAY, & array)) {
+               warning(ext_id, _("reada: second argument is not an array"));
+               errno = EINVAL;
+               update_ERRNO_int(errno);
+               make_number(0.0, result);
+               return result;
+       }
+       return read_backend(result, array.array_cookie, "read", read_one);
+}
+
+/* do_readall --- read top-level variables */
+
+static awk_value_t *
+do_readall(int nargs, awk_value_t *result, struct awk_ext_func *unused)
+{
+       return read_backend(result, NULL, "readall", read_global);
+}
+
 
 /* read_array --- read in an array or sub-array */
 
@@ -466,7 +627,7 @@ read_array(FILE *fp, awk_array_t array)
        count = ntohl(count);
 
        for (i = 0; i < count; i++) {
-               if (read_elem(fp, & new_elem)) {
+               if (read_elem(fp, & new_elem, regular_array_handle)) {
                        /* add to array */
                        if (! set_array_element_by_elem(array, & new_elem)) {
                                warning(ext_id, _("read_array: 
set_array_element failed"));
@@ -485,7 +646,7 @@ read_array(FILE *fp, awk_array_t array)
 /* read_elem --- read in a single element */
 
 static awk_bool_t
-read_elem(FILE *fp, awk_element_t *element)
+read_elem(FILE *fp, awk_element_t *element, array_handle_t array_handle)
 {
        uint32_t index_len;
        static char *buffer;
@@ -523,7 +684,7 @@ read_elem(FILE *fp, awk_element_t *element)
                make_null_string(& element->index);
        }
 
-       if (! read_value(fp, & element->value))
+       if (! read_value(fp, & element->value, array_handle, & element->index))
                return awk_false;
 
        return awk_true;
@@ -532,7 +693,7 @@ read_elem(FILE *fp, awk_element_t *element)
 /* read_value --- read a number or a string */
 
 static awk_bool_t
-read_value(FILE *fp, awk_value_t *value)
+read_value(FILE *fp, awk_value_t *value, array_handle_t array_handle, 
awk_value_t *idx)
 {
        uint32_t code, len;
 
@@ -542,7 +703,7 @@ read_value(FILE *fp, awk_value_t *value)
        code = ntohl(code);
 
        if (code == VT_ARRAY) {
-               awk_array_t array = create_array();
+               awk_array_t array = (*array_handle)(idx);
 
                if (! read_array(fp, array))
                        return awk_false;
@@ -662,6 +823,8 @@ read_number(FILE *fp, awk_value_t *value, uint32_t code)
 static awk_ext_func_t func_table[] = {
        { "writea", do_writea, 2, 2, awk_false, NULL },
        { "reada", do_reada, 2, 2, awk_false, NULL },
+       { "writeall", do_writeall, 1, 1, awk_false, NULL },
+       { "readall", do_readall, 1, 1, awk_false, NULL },
 };
 
 
diff --git a/pc/ChangeLog b/pc/ChangeLog
index 951b7d5..675ebbf 100644
--- a/pc/ChangeLog
+++ b/pc/ChangeLog
@@ -2,6 +2,10 @@
 
        * Makefile.tst: Regenerated.
 
+2021-12-08         Andrew J. Schorr      <aschorr@telemetry-investments.com>
+
+       * Makefile.tst: Regenerated.
+
 2021-12-01         Arnold D. Robbins     <arnold@skeeve.com>
 
        * gawkmisc.pc (os_maybe_set_errno): Renamed from
diff --git a/pc/Makefile.tst b/pc/Makefile.tst
index a25aec7..4d4b125 100644
--- a/pc/Makefile.tst
+++ b/pc/Makefile.tst
@@ -233,6 +233,7 @@ SHLIB_TESTS = \
        getfile \
        inplace1 inplace2 inplace2bcomp inplace3 inplace3bcomp \
        ordchr ordchr2 \
+       readall \
        readdir readdir_test readdir_retest readfile readfile2 revout \
        revtwoway rwarray \
        testext time
@@ -345,7 +346,8 @@ EXPECTED_FAIL_ZOS = \
 GENTESTS_UNUSED = Makefile.in checknegtime.awk dtdgport.awk fix-fmtspcl.awk \
        fmtspcl-mpfr.ok fmtspcl.awk fmtspcl.tok gtlnbufv.awk hello.awk \
        inchello.awk inclib.awk inplace.1.in inplace.2.in inplace.in \
-       printfloat.awk readdir0.awk valgrind.awk xref.awk
+       printfloat.awk readdir0.awk valgrind.awk xref.awk \
+       readall1.awk readall2.awk
 
 
 # List of tests on MinGW or DJGPP that need a different cmp program
@@ -1063,6 +1065,13 @@ readdir_retest:
        @-$(AWK) -lreaddir_test -F$(SLASH) -f "$(srcdir)"/$@.awk 
"$(top_srcdir)" > _$@
        @-$(CMP) $@.ok _$@ && rm -f $@.ok _$@
 
+readall:
+       @echo $@
+       @-$(AWK) -lrwarray -f "$(srcdir)"/$@1.awk -v "ofile=readall.state" > _$@
+       @-$(AWK) -lrwarray -f "$(srcdir)"/$@2.awk -v "ifile=readall.state" >> 
_$@
+       @-$(CMP) $@.ok _$@ && rm -f _$@
+       @-$(RM) -f readall.state
+
 fts:
        @echo $@
        @echo Expect $@ to fail with MinGW.
diff --git a/test/ChangeLog b/test/ChangeLog
index 6763764..c6c0c65 100644
--- a/test/ChangeLog
+++ b/test/ChangeLog
@@ -1,5 +1,14 @@
 2021-12-08         Andrew J. Schorr      <aschorr@telemetry-investments.com>
 
+       * Makefile.am (EXTRA_DIST): Add readall1.awk, readall2.awk, and
+       readall.ok.
+       (SHLIB_TESTS): Add readall.
+       (GENTESTS_UNUSED): Add readall1.awk and readall2.awk.
+       (readall): Add new test of the writeall and readall functions.
+       * readall1.awk, readall2.awk, readall.ok: New files.
+
+2021-12-08         Andrew J. Schorr      <aschorr@telemetry-investments.com>
+
        * Makefile.am (testext): Change awk pattern to include functions in
        testext.awk.
        (diffout): If running MPFR tests and there's an mpfr.ok file, compare
diff --git a/test/Makefile.am b/test/Makefile.am
index f9cdfd4..1fa8ad3 100644
--- a/test/Makefile.am
+++ b/test/Makefile.am
@@ -1031,6 +1031,9 @@ EXTRA_DIST = \
        range1.ok \
        range2.awk \
        range2.ok \
+       readall1.awk \
+       readall2.awk \
+       readall.ok \
        readbuf.awk \
        readbuf.ok \
        readdir.awk \
@@ -1490,6 +1493,7 @@ SHLIB_TESTS = \
        getfile \
        inplace1 inplace2 inplace2bcomp inplace3 inplace3bcomp \
        ordchr ordchr2 \
+       readall \
        readdir readdir_test readdir_retest readfile readfile2 revout \
        revtwoway rwarray \
        testext time
@@ -1597,7 +1601,8 @@ ZOS_FAIL = @ZOS_FAIL@
 GENTESTS_UNUSED = Makefile.in checknegtime.awk dtdgport.awk fix-fmtspcl.awk \
        fmtspcl-mpfr.ok fmtspcl.awk fmtspcl.tok gtlnbufv.awk hello.awk \
        inchello.awk inclib.awk inplace.1.in inplace.2.in inplace.in \
-       printfloat.awk readdir0.awk valgrind.awk xref.awk
+       printfloat.awk readdir0.awk valgrind.awk xref.awk \
+       readall1.awk readall2.awk
 
 # List of tests on MinGW or DJGPP that need a different cmp program
 NEED_TESTOUTCMP = \
@@ -2317,6 +2322,13 @@ readdir_retest:
        @-$(AWK) -lreaddir_test -F/ -f "$(srcdir)"/$@.awk "$(top_srcdir)" > _$@
        @-$(CMP) $@.ok _$@ && rm -f $@.ok _$@
 
+readall:
+       @echo $@
+       @-$(AWK) -lrwarray -f "$(srcdir)"/$@1.awk -v "ofile=readall.state" > _$@
+       @-$(AWK) -lrwarray -f "$(srcdir)"/$@2.awk -v "ifile=readall.state" >> 
_$@
+       @-$(CMP) $@.ok _$@ && rm -f _$@
+       @-$(RM) -f readall.state
+
 fts:
        @echo $@
        @-case `uname` in \
diff --git a/test/Makefile.in b/test/Makefile.in
index a4d2758..c6c131f 100644
--- a/test/Makefile.in
+++ b/test/Makefile.in
@@ -1297,6 +1297,9 @@ EXTRA_DIST = \
        range1.ok \
        range2.awk \
        range2.ok \
+       readall1.awk \
+       readall2.awk \
+       readall.ok \
        readbuf.awk \
        readbuf.ok \
        readdir.awk \
@@ -1752,6 +1755,7 @@ SHLIB_TESTS = \
        getfile \
        inplace1 inplace2 inplace2bcomp inplace3 inplace3bcomp \
        ordchr ordchr2 \
+       readall \
        readdir readdir_test readdir_retest readfile readfile2 revout \
        revtwoway rwarray \
        testext time
@@ -1864,7 +1868,8 @@ EXPECTED_FAIL_ZOS = \
 GENTESTS_UNUSED = Makefile.in checknegtime.awk dtdgport.awk fix-fmtspcl.awk \
        fmtspcl-mpfr.ok fmtspcl.awk fmtspcl.tok gtlnbufv.awk hello.awk \
        inchello.awk inclib.awk inplace.1.in inplace.2.in inplace.in \
-       printfloat.awk readdir0.awk valgrind.awk xref.awk
+       printfloat.awk readdir0.awk valgrind.awk xref.awk \
+       readall1.awk readall2.awk
 
 
 # List of tests on MinGW or DJGPP that need a different cmp program
@@ -2771,6 +2776,13 @@ readdir_retest:
        @-$(AWK) -lreaddir_test -F/ -f "$(srcdir)"/$@.awk "$(top_srcdir)" > _$@
        @-$(CMP) $@.ok _$@ && rm -f $@.ok _$@
 
+readall:
+       @echo $@
+       @-$(AWK) -lrwarray -f "$(srcdir)"/$@1.awk -v "ofile=readall.state" > _$@
+       @-$(AWK) -lrwarray -f "$(srcdir)"/$@2.awk -v "ifile=readall.state" >> 
_$@
+       @-$(CMP) $@.ok _$@ && rm -f _$@
+       @-$(RM) -f readall.state
+
 fts:
        @echo $@
        @-case `uname` in \
diff --git a/test/readall.ok b/test/readall.ok
new file mode 100644
index 0000000..b343af5
--- /dev/null
+++ b/test/readall.ok
@@ -0,0 +1,7 @@
+1
+1
+5.9 3 -2.327
+zebra[archie] = banana
+zebra[0] = apple
+zebra[3][foo] = bar
+zebra[3][bar] = foo
diff --git a/test/readall1.awk b/test/readall1.awk
new file mode 100644
index 0000000..2888d15
--- /dev/null
+++ b/test/readall1.awk
@@ -0,0 +1,10 @@
+BEGIN {
+       x = 5.9
+       y = 3
+       z = -2.327
+       zebra[0] = "apple"
+       zebra["archie"] = "banana"
+       zebra[3]["foo"] = "bar"
+       zebra[3]["bar"] = "foo"
+       print writeall(ofile)
+}
diff --git a/test/readall2.awk b/test/readall2.awk
new file mode 100644
index 0000000..8b79849
--- /dev/null
+++ b/test/readall2.awk
@@ -0,0 +1,15 @@
+function printarray(n, x,   i) {
+       for (i in x) {
+               if (isarray(x[i]))
+                       printarray((n "[" i "]"), x[i])
+               else
+                       printf "%s[%s] = %s\n", n, i, x[i]
+       }
+}
+
+BEGIN {
+       print readall(ifile)
+       print x, y, z
+       #print zebra[0], zebra[3]["foo"], zebra[3]["bar"]
+       printarray("zebra", zebra)
+}

-----------------------------------------------------------------------

Summary of changes:
 doc/ChangeLog         |   5 +
 doc/gawk.info         | 205 ++++++++++++++++++++++-----------------
 doc/gawk.texi         |  34 ++++++-
 doc/gawktexi.in       |  34 ++++++-
 doc/wordlist          |   2 +
 extension/ChangeLog   |  38 ++++++++
 extension/rwarray.3am |  35 ++++++-
 extension/rwarray.c   | 263 ++++++++++++++++++++++++++++++++++++++++----------
 pc/ChangeLog          |   4 +
 pc/Makefile.tst       |  11 ++-
 test/ChangeLog        |   9 ++
 test/Makefile.am      |  14 ++-
 test/Makefile.in      |  14 ++-
 test/readall.ok       |   7 ++
 test/readall1.awk     |  10 ++
 test/readall2.awk     |  15 +++
 16 files changed, 551 insertions(+), 149 deletions(-)
 create mode 100644 test/readall.ok
 create mode 100644 test/readall1.awk
 create mode 100644 test/readall2.awk


hooks/post-receive
-- 
gawk



reply via email to

[Prev in Thread] Current Thread [Next in Thread]