[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[SCM] gawk branch, master, updated. gawk-4.1.0-5782-gb7300af5
|
From: |
Arnold Robbins |
|
Subject: |
[SCM] gawk branch, master, updated. gawk-4.1.0-5782-gb7300af5 |
|
Date: |
Mon, 10 Feb 2025 03:35:14 -0500 (EST) |
This is an automated email from the git hooks/post-receive script. It was
generated because a ref change was pushed to the repository containing
the project "gawk".
The branch, master has been updated
via b7300af55606459dfe2d217188bc85c493f9d31a (commit)
via cfdf8297dac3a00ecef2483ef7add869483b0e26 (commit)
from 06f6b279ccbc321259f8a5067d8271e5079717d4 (commit)
Those revisions listed above that are new to this repository have
not appeared on any other notification email; so we list those
revisions in full, below.
- Log -----------------------------------------------------------------
http://git.sv.gnu.org/cgit/gawk.git/commit/?id=b7300af55606459dfe2d217188bc85c493f9d31a
commit b7300af55606459dfe2d217188bc85c493f9d31a
Merge: 06f6b279 cfdf8297
Author: Arnold D. Robbins <arnold@skeeve.com>
Date: Mon Feb 10 10:35:02 2025 +0200
Merge branch 'gawk-5.3-stable'
diff --cc doc/gawk.info
index ab7836a5,a7630d9a..ab6ffea4
--- a/doc/gawk.info
+++ b/doc/gawk.info
@@@ -36563,10 -36457,8 +36566,12 @@@ Inde
(line 9)
* @ (at-sign), @namespace directive, BEGIN, BEGINFILE, END, ENDFILE and:
Changing The Namespace.
(line 37)
+* @ (at-sign), @nsinclude directive: Include Files. (line 100)
+* @ (at-sign), @nsinclude directive <1>: Inclusion For Namespaces.
+ (line 6)
+* @ (at-sign), @nsinclude directive <2>: Feature History. (line 543)
+ * @ (at-sign), strongly typed regexp constant: Strong Regexp Constants.
+ (line 24)
* * (asterisk), * operator, as multiplication operator: Precedence.
(line 54)
* * (asterisk), * operator, as regexp operator: Regexp Operator Details.
@@@ -36912,10 -36804,8 +36917,12 @@@
(line 9)
* at-sign (@), @namespace directive, BEGIN, BEGINFILE, END, ENDFILE and:
Changing The Namespace.
(line 37)
+* at-sign (@), @nsinclude directive: Include Files. (line 100)
+* at-sign (@), @nsinclude directive <1>: Inclusion For Namespaces.
+ (line 6)
+* at-sign (@), @nsinclude directive <2>: Feature History. (line 543)
+ * at-sign (@), strongly typed regexp constant: Strong Regexp Constants.
+ (line 24)
* atan2: Numeric Functions. (line 11)
* automatic displays, in debugger: Debugger Info. (line 24)
* awf (amazingly workable formatter) program: Glossary. (line 23)
@@@ -40149,630 -40027,629 +40156,630 @@@
Tag Table:
Node: Top1203
-Node: Foreword347051
-Node: Foreword451625
-Node: Preface53169
-Ref: Preface-Footnote-156155
-Ref: Preface-Footnote-256264
-Ref: Preface-Footnote-356498
-Node: History56644
-Node: Names59258
-Ref: Names-Footnote-160412
-Node: This Manual60575
-Ref: This Manual-Footnote-167505
-Node: Conventions67613
-Node: Manual History70042
-Ref: Manual History-Footnote-173067
-Ref: Manual History-Footnote-273108
-Node: How To Contribute73182
-Node: Acknowledgments74128
-Node: Getting Started79117
-Node: Running gawk81644
-Node: One-shot82862
-Node: Read Terminal84161
-Node: Long86215
-Node: Executable Scripts87865
-Ref: Executable Scripts-Footnote-190635
-Node: Comments90742
-Node: Quoting93272
-Node: DOS Quoting98916
-Node: Sample Data Files100996
-Node: Very Simple103631
-Node: Two Rules109885
-Node: More Complex111837
-Node: Statements/Lines114257
-Ref: Statements/Lines-Footnote-1119769
-Node: Other Features120058
-Node: When121026
-Ref: When-Footnote-1122828
-Node: Intro Summary122893
-Node: Invoking Gawk123847
-Node: Command Line125409
-Node: Options126259
-Ref: Options-Footnote-1145601
-Ref: Options-Footnote-2145842
-Node: Other Arguments145867
-Node: Naming Standard Input150032
-Node: Environment Variables151302
-Node: AWKPATH Variable151876
-Ref: AWKPATH Variable-Footnote-1155460
-Ref: AWKPATH Variable-Footnote-2155494
-Node: AWKLIBPATH Variable155885
-Ref: AWKLIBPATH Variable-Footnote-1157660
-Node: Other Environment Variables158055
-Node: Exit Status162531
-Node: Include Files163244
-Node: Loading Shared Libraries167304
-Node: Obsolete168796
-Node: Undocumented169544
-Node: Invoking Summary169841
-Node: Regexp172866
-Node: Regexp Usage174360
-Node: Escape Sequences176461
-Ref: Escape Sequences-Footnote-1184214
-Node: Regexp Operators184292
-Node: Regexp Operator Details184785
-Ref: Regexp Operator Details-Footnote-1192796
-Node: Interval Expressions192955
-Ref: Interval Expressions-Footnote-1195222
-Node: Bracket Expressions195320
-Ref: table-char-classes198069
-Ref: Bracket Expressions-Footnote-1201603
-Node: Leftmost Longest201872
-Node: Computed Regexps203228
-Node: GNU Regexp Operators206743
-Node: Case-sensitivity211840
-Ref: Case-sensitivity-Footnote-1214790
-Ref: Case-sensitivity-Footnote-2215033
-Node: Regexp Summary215145
-Node: Reading Files216667
-Node: Records218986
-Node: awk split records220261
-Node: gawk split records225143
-Ref: gawk split records-Footnote-1230431
-Node: Fields230468
-Ref: Fields-Footnote-1233384
-Node: Nonconstant Fields233504
-Ref: Nonconstant Fields-Footnote-1235976
-Node: Changing Fields236192
-Node: Field Separators242490
-Node: Default Field Splitting245359
-Node: Regexp Field Splitting246501
-Node: Single Character Fields250330
-Node: Comma Separated Fields251419
-Ref: table-csv-examples252829
-Node: Command Line Field Separator255134
-Node: Full Line Fields258510
-Ref: Full Line Fields-Footnote-1260088
-Ref: Full Line Fields-Footnote-2260134
-Node: Field Splitting Summary260239
-Node: Constant Size262556
-Node: Fixed width data263300
-Node: Skipping intervening266815
-Node: Allowing trailing data267617
-Node: Fields with fixed data268678
-Node: Splitting By Content270300
-Ref: Splitting By Content-Footnote-1275801
-Node: FS versus FPAT275966
-Node: Testing field creation277149
-Node: Multiple Line278923
-Node: Getline285392
-Node: Plain Getline287980
-Node: Getline/Variable290628
-Node: Getline/File291824
-Node: Getline/Variable/File293272
-Ref: Getline/Variable/File-Footnote-1294917
-Node: Getline/Pipe295013
-Node: Getline/Variable/Pipe297821
-Node: Getline/Coprocess299004
-Node: Getline/Variable/Coprocess300327
-Node: Getline Notes301093
-Node: Getline Summary305070
-Ref: table-getline-variants305514
-Node: Read Timeout306418
-Ref: Read Timeout-Footnote-1310380
-Node: Retrying Input310438
-Node: Command-line directories311705
-Node: Input Summary312643
-Node: Input Exercises316023
-Node: Printing316461
-Node: Print318404
-Node: Print Examples319905
-Node: Output Separators322750
-Node: OFMT324857
-Node: Printf326570
-Node: Basic Printf327375
-Node: Control Letters329010
-Node: Format Modifiers334456
-Node: Printf Examples340728
-Node: Redirection343269
-Node: Special FD351041
-Ref: Special FD-Footnote-1354339
-Node: Special Files354417
-Node: Other Inherited Files355046
-Node: Special Network356111
-Node: Special Caveats356999
-Node: Close Files And Pipes357982
-Ref: Close Files And Pipes-Footnote-1364106
-Node: Close Return Value364254
-Ref: table-close-pipe-return-values365525
-Ref: Close Return Value-Footnote-1366356
-Node: Noflush366512
-Node: Nonfatal368020
-Node: Output Summary370435
-Node: Output Exercises371721
-Node: Expressions372412
-Node: Values373612
-Node: Constants374290
-Node: Scalar Constants374985
-Ref: Scalar Constants-Footnote-1377561
-Ref: Scalar Constants-Footnote-2377811
-Node: Nondecimal-numbers377891
-Node: Regexp Constants381004
-Node: Using Constant Regexps381550
-Node: Standard Regexp Constants382196
-Node: Strong Regexp Constants385492
-Node: Variables389335
-Node: Using Variables390000
-Node: Assignment Options391974
-Node: Conversion394525
-Node: Strings And Numbers395057
-Ref: Strings And Numbers-Footnote-1398267
-Node: Locale influences conversions398376
-Ref: table-locale-affects401214
-Node: All Operators401856
-Node: Arithmetic Ops402497
-Node: Concatenation405320
-Ref: Concatenation-Footnote-1408256
-Node: Assignment Ops408375
-Ref: table-assign-ops413502
-Node: Increment Ops414883
-Node: Truth Values and Conditions418474
-Node: Truth Values419568
-Node: Typing and Comparison420648
-Node: Variable Typing421480
-Ref: Variable Typing-Footnote-1428124
-Ref: Variable Typing-Footnote-2428204
-Node: Comparison Operators428285
-Ref: table-relational-ops428712
-Node: POSIX String Comparison432388
-Ref: POSIX String Comparison-Footnote-1434145
-Ref: POSIX String Comparison-Footnote-2434288
-Node: Boolean Ops434373
-Ref: Boolean Ops-Footnote-1439047
-Node: Conditional Exp439143
-Node: Function Calls440923
-Node: Precedence444870
-Node: Locales448733
-Node: Expressions Summary450409
-Node: Patterns and Actions453064
-Node: Pattern Overview454200
-Node: Regexp Patterns455925
-Node: Expression Patterns456471
-Node: Ranges460376
-Node: BEGIN/END463550
-Node: Using BEGIN/END464359
-Ref: Using BEGIN/END-Footnote-1467267
-Node: I/O And BEGIN/END467377
-Node: BEGINFILE/ENDFILE469859
-Node: Empty473289
-Node: Using Shell Variables473606
-Node: Action Overview475942
-Node: Statements478378
-Node: If Statement480274
-Node: While Statement481837
-Node: Do Statement483925
-Node: For Statement485109
-Node: Switch Statement488464
-Node: Break Statement491013
-Node: Continue Statement493205
-Node: Next Statement495136
-Node: Nextfile Statement497615
-Node: Exit Statement500469
-Node: Built-in Variables502996
-Node: User-modified504173
-Node: Auto-set512380
-Ref: Auto-set-Footnote-1530464
-Ref: Auto-set-Footnote-2530682
-Node: ARGC and ARGV530738
-Node: Pattern Action Summary535167
-Node: Arrays537773
-Node: Array Basics539146
-Node: Array Intro539994
-Ref: figure-array-elements542005
-Ref: Array Intro-Footnote-1544858
-Node: Reference to Elements544990
-Node: Assigning Elements547510
-Node: Array Example548005
-Node: Scanning an Array549967
-Node: Controlling Scanning553062
-Ref: Controlling Scanning-Footnote-1559697
-Node: Numeric Array Subscripts560021
-Node: Uninitialized Subscripts562289
-Node: Delete563962
-Ref: Delete-Footnote-1566775
-Node: Multidimensional566832
-Node: Multiscanning570035
-Node: Arrays of Arrays571702
-Node: Arrays Summary576006
-Node: Functions578193
-Node: Built-in579395
-Node: Calling Built-in580584
-Node: Boolean Functions582624
-Node: Numeric Functions583186
-Ref: Numeric Functions-Footnote-1587371
-Ref: Numeric Functions-Footnote-2588054
-Ref: Numeric Functions-Footnote-3588106
-Node: String Functions588382
-Ref: String Functions-Footnote-1614833
-Ref: String Functions-Footnote-2614965
-Ref: String Functions-Footnote-3615221
-Node: Gory Details615308
-Ref: table-sub-escapes617317
-Ref: table-sub-proposed618869
-Ref: table-posix-sub620343
-Ref: table-gensub-escapes621995
-Ref: Gory Details-Footnote-1622823
-Node: I/O Functions622977
-Ref: table-system-return-values629654
-Ref: I/O Functions-Footnote-1631816
-Ref: I/O Functions-Footnote-2631964
-Node: Time Functions632084
-Ref: Time Functions-Footnote-1643796
-Ref: Time Functions-Footnote-2643864
-Ref: Time Functions-Footnote-3644026
-Ref: Time Functions-Footnote-4644137
-Ref: Time Functions-Footnote-5644253
-Ref: Time Functions-Footnote-6644480
-Node: Bitwise Functions644758
-Ref: table-bitwise-ops645356
-Ref: Bitwise Functions-Footnote-1651598
-Ref: Bitwise Functions-Footnote-2651775
-Node: Type Functions651970
-Node: I18N Functions654141
-Node: User-defined655876
-Node: Definition Syntax656622
-Ref: Definition Syntax-Footnote-1662477
-Node: Function Example662552
-Ref: Function Example-Footnote-1665531
-Node: Function Calling665553
-Node: Calling A Function666145
-Node: Variable Scope667115
-Node: Pass By Value/Reference670169
-Node: Function Caveats672897
-Ref: Function Caveats-Footnote-1674988
-Node: Return Statement675108
-Node: Dynamic Typing678140
-Node: Dynamic Typing Awk678722
-Node: Dynamic Typing Gawk680860
-Node: Shadowed Variables684230
-Node: Indirect Calls686578
-Node: Functions Summary697719
-Node: Library Functions700685
-Ref: Library Functions-Footnote-1704233
-Ref: Library Functions-Footnote-2704378
-Node: Library Names704553
-Ref: Library Names-Footnote-1708324
-Ref: Library Names-Footnote-2708551
-Node: General Functions708645
-Node: Strtonum Function709915
-Node: Assert Function712997
-Node: Round Function716552
-Node: Cliff Random Function718124
-Node: Ordinal Functions719149
-Ref: Ordinal Functions-Footnote-1722252
-Ref: Ordinal Functions-Footnote-2722504
-Node: Join Function722718
-Ref: Join Function-Footnote-1724516
-Node: Getlocaltime Function724720
-Node: Readfile Function728494
-Node: Shell Quoting730523
-Node: Isnumeric Function731979
-Node: To CSV Function733415
-Node: Data File Management735507
-Node: Filetrans Function736139
-Node: Rewind Function740415
-Node: File Checking742386
-Ref: File Checking-Footnote-1743752
-Node: Empty Files743957
-Node: Ignoring Assigns746020
-Node: Getopt Function747594
-Ref: Getopt Function-Footnote-1763412
-Node: Passwd Functions763624
-Ref: Passwd Functions-Footnote-1772759
-Node: Group Functions772847
-Ref: Group Functions-Footnote-1780971
-Node: Walking Arrays781182
-Node: Library Functions Summary784228
-Node: Library Exercises785648
-Node: Sample Programs786133
-Node: Running Examples786915
-Node: Clones787667
-Node: Cut Program788935
-Node: Egrep Program799359
-Node: Id Program808658
-Node: Split Program818750
-Ref: Split Program-Footnote-1828963
-Node: Tee Program829148
-Node: Uniq Program832054
-Node: Wc Program839914
-Node: Bytes vs. Characters840309
-Node: Using extensions841909
-Node: wc program842687
-Node: Miscellaneous Programs847680
-Node: Dupword Program848905
-Node: Alarm Program850954
-Node: Translate Program855857
-Ref: Translate Program-Footnote-1860566
-Node: Labels Program860844
-Ref: Labels Program-Footnote-1864279
-Node: Word Sorting864363
-Node: History Sorting868537
-Node: Extract Program870810
-Node: Simple Sed879063
-Node: Igawk Program882273
-Ref: Igawk Program-Footnote-1897483
-Ref: Igawk Program-Footnote-2897689
-Ref: Igawk Program-Footnote-3897819
-Node: Anagram Program897946
-Node: Signature Program901032
-Node: Programs Summary902283
-Node: Programs Exercises903537
-Ref: Programs Exercises-Footnote-1908366
-Node: Advanced Features908452
-Node: Nondecimal Data910933
-Node: Boolean Typed Values912563
-Node: Array Sorting914520
-Node: Controlling Array Traversal915249
-Ref: Controlling Array Traversal-Footnote-1923752
-Node: Array Sorting Functions923874
-Ref: Array Sorting Functions-Footnote-1929971
-Node: Two-way I/O930179
-Ref: Two-way I/O-Footnote-1938150
-Ref: Two-way I/O-Footnote-2938341
-Node: TCP/IP Networking938423
-Node: Profiling941591
-Node: Persistent Memory951261
-Ref: Persistent Memory-Footnote-1960701
-Node: Extension Philosophy960832
-Node: Advanced Features Summary962359
-Node: Internationalization964625
-Node: I18N and L10N966327
-Node: Explaining gettext967022
-Ref: Explaining gettext-Footnote-1973158
-Ref: Explaining gettext-Footnote-2973351
-Node: Programmer i18n973516
-Ref: Programmer i18n-Footnote-1978628
-Node: Translator i18n978677
-Node: String Extraction979507
-Ref: String Extraction-Footnote-1980683
-Node: Printf Ordering980781
-Ref: Printf Ordering-Footnote-1983639
-Node: I18N Portability983707
-Ref: I18N Portability-Footnote-1986267
-Node: I18N Example986334
-Ref: I18N Example-Footnote-1989728
-Ref: I18N Example-Footnote-2989801
-Node: Gawk I18N989918
-Node: I18N Summary990572
-Node: Debugger991969
-Node: Debugging992989
-Node: Debugging Concepts993438
-Node: Debugging Terms995255
-Node: Awk Debugging997858
-Ref: Awk Debugging-Footnote-1998831
-Node: Sample Debugging Session998967
-Node: Debugger Invocation999517
-Node: Finding The Bug1001142
-Node: List of Debugger Commands1007774
-Node: Breakpoint Control1009151
-Node: Debugger Execution Control1012973
-Node: Viewing And Changing Data1016447
-Node: Execution Stack1020181
-Node: Debugger Info1021862
-Node: Miscellaneous Debugger Commands1026157
-Node: Readline Support1031398
-Node: Limitations1032336
-Node: Debugging Summary1034960
-Node: Namespaces1036259
-Node: Global Namespace1037386
-Node: Qualified Names1038820
-Node: Default Namespace1039855
-Node: Changing The Namespace1040628
-Node: Naming Rules1042310
-Node: Internal Name Management1044265
-Node: Namespace Example1045335
-Node: Namespace And Features1047912
-Node: Namespace Summary1049367
-Node: Arbitrary Precision Arithmetic1050878
-Node: Computer Arithmetic1052397
-Ref: table-numeric-ranges1056322
-Ref: table-floating-point-ranges1056819
-Ref: Computer Arithmetic-Footnote-11057477
-Node: Math Definitions1057534
-Ref: table-ieee-formats1060566
-Node: MPFR features1061139
-Node: MPFR On Parole1061592
-Ref: MPFR On Parole-Footnote-11062433
-Node: MPFR Intro1062592
-Node: FP Math Caution1064277
-Ref: FP Math Caution-Footnote-11065349
-Node: Inexactness of computations1065723
-Node: Inexact representation1066754
-Node: Comparing FP Values1068135
-Node: Errors accumulate1069393
-Node: Strange values1070858
-Ref: Strange values-Footnote-11073512
-Node: Getting Accuracy1073617
-Node: Try To Round1076354
-Node: Setting precision1077261
-Ref: table-predefined-precision-strings1077966
-Node: Setting the rounding mode1079850
-Ref: table-gawk-rounding-modes1080232
-Ref: Setting the rounding mode-Footnote-11084284
-Node: Arbitrary Precision Integers1084469
-Ref: Arbitrary Precision Integers-Footnote-11087679
-Node: Checking for MPFR1087833
-Node: POSIX Floating Point Problems1089323
-Ref: POSIX Floating Point Problems-Footnote-11094143
-Node: Floating point summary1094181
-Node: Dynamic Extensions1096437
-Node: Extension Intro1098034
-Node: Plugin License1099336
-Node: Extension Mechanism Outline1100149
-Ref: figure-load-extension1100600
-Ref: figure-register-new-function1102178
-Ref: figure-call-new-function1103287
-Node: Extension API Description1105402
-Node: Extension API Functions Introduction1107131
-Ref: table-api-std-headers1109025
-Node: General Data Types1113466
-Ref: General Data Types-Footnote-11122612
-Node: Memory Allocation Functions1122915
-Ref: Memory Allocation Functions-Footnote-11127630
-Node: Constructor Functions1127729
-Node: API Ownership of MPFR and GMP Values1131630
-Node: Registration Functions1133183
-Node: Extension Functions1133887
-Node: Exit Callback Functions1139461
-Node: Extension Version String1140775
-Node: Input Parsers1141470
-Node: Output Wrappers1156089
-Node: Two-way processors1160931
-Node: Printing Messages1163284
-Ref: Printing Messages-Footnote-11164495
-Node: Updating ERRNO1164648
-Node: Requesting Values1165447
-Ref: table-value-types-returned1166200
-Node: Accessing Parameters1168259
-Node: Symbol Table Access1169540
-Node: Symbol table by name1170052
-Ref: Symbol table by name-Footnote-11173253
-Node: Symbol table by cookie1173385
-Ref: Symbol table by cookie-Footnote-11177654
-Node: Cached values1177718
-Ref: Cached values-Footnote-11181350
-Node: Array Manipulation1181507
-Ref: Array Manipulation-Footnote-11182606
-Node: Array Data Types1182643
-Ref: Array Data Types-Footnote-11185461
-Node: Array Functions1185557
-Node: Flattening Arrays1190586
-Node: Creating Arrays1197634
-Node: Redirection API1202476
-Node: Extension API Variables1205493
-Node: Extension Versioning1206216
-Ref: gawk-api-version1206645
-Node: Extension GMP/MPFR Versioning1208432
-Node: Extension API Informational Variables1210136
-Node: Extension API Boilerplate1211389
-Node: Changes from API V11215519
-Node: Finding Extensions1217151
-Node: Extension Example1217726
-Node: Internal File Description1218548
-Node: Internal File Ops1222840
-Ref: Internal File Ops-Footnote-11234390
-Node: Using Internal File Ops1234538
-Ref: Using Internal File Ops-Footnote-11236971
-Node: Extension Samples1237249
-Node: Extension Sample File Functions1238818
-Node: Extension Sample Fnmatch1246943
-Node: Extension Sample Fork1248538
-Node: Extension Sample Inplace1249814
-Node: Extension Sample Ord1253916
-Node: Extension Sample Readdir1254792
-Ref: table-readdir-file-types1255581
-Node: Extension Sample Revout1256937
-Node: Extension Sample Rev2way1257534
-Node: Extension Sample Read write array1258286
-Node: Extension Sample Readfile1261560
-Node: Extension Sample Time1262691
-Node: Extension Sample API Tests1264691
-Node: gawkextlib1265199
-Node: Extension summary1268404
-Node: Extension Exercises1272252
-Node: Language History1273522
-Node: V7/SVR3.11275234
-Node: SVR41277584
-Node: POSIX1279116
-Node: BTL1280541
-Node: POSIX/GNU1281308
-Ref: Gawk Extension Functions1284698
-Node: Feature History1288122
-Node: Common Extensions1308024
-Node: Ranges and Locales1309499
-Ref: Ranges and Locales-Footnote-11314284
-Ref: Ranges and Locales-Footnote-21314311
-Ref: Ranges and Locales-Footnote-31314546
-Node: Contributors1314769
-Node: History summary1321121
-Node: Installation1322563
-Node: Gawk Distribution1323527
-Node: Getting1324019
-Node: Extracting1325018
-Node: Distribution contents1326724
-Node: Unix Installation1334614
-Node: Quick Installation1335434
-Node: Compiling with MPFR1337974
-Node: Shell Startup Files1338680
-Node: Additional Configuration Options1339837
-Node: Configuration Philosophy1342220
-Node: Compiling from Git1344720
-Node: Building the Documentation1345281
-Node: Non-Unix Installation1346693
-Node: PC Installation1347169
-Node: PC Binary Installation1348038
-Node: PC Compiling1348931
-Node: PC Using1350109
-Node: Cygwin1353825
-Node: MSYS1355078
-Node: OpenVMS Installation1355704
-Node: OpenVMS Compilation1356385
-Ref: OpenVMS Compilation-Footnote-11357868
-Node: OpenVMS Dynamic Extensions1357926
-Node: OpenVMS Installation Details1359562
-Node: OpenVMS Running1361993
-Node: OpenVMS GNV1366130
-Node: Bugs1366885
-Node: Bug definition1367805
-Node: Bug address1371407
-Node: Usenet1374976
-Node: Performance bugs1376189
-Node: Asking for help1379193
-Node: Maintainers1381180
-Node: Other Versions1382207
-Node: Installation summary1392275
-Node: Notes1393657
-Node: Compatibility Mode1394467
-Node: Additions1395289
-Node: Accessing The Source1396234
-Node: Adding Code1397767
-Node: New Ports1404878
-Node: Derived Files1409381
-Ref: Derived Files-Footnote-11415192
-Ref: Derived Files-Footnote-21415227
-Ref: Derived Files-Footnote-31415838
-Node: Future Extensions1415952
-Node: Implementation Limitations1416622
-Node: Extension Design1417864
-Node: Old Extension Problems1419024
-Ref: Old Extension Problems-Footnote-11420596
-Node: Extension New Mechanism Goals1420657
-Ref: Extension New Mechanism Goals-Footnote-11424127
-Node: Extension Other Design Decisions1424328
-Node: Extension Future Growth1426525
-Node: Notes summary1427145
-Node: Basic Concepts1428355
-Node: Basic High Level1429040
-Ref: figure-general-flow1429322
-Ref: figure-process-flow1430024
-Ref: Basic High Level-Footnote-11433394
-Node: Basic Data Typing1433583
-Node: Glossary1436991
-Node: Copying1469873
-Node: GNU Free Documentation License1507431
-Node: Index1532554
+Node: Foreword347180
+Node: Foreword451754
+Node: Preface53298
+Ref: Preface-Footnote-156284
+Ref: Preface-Footnote-256393
+Ref: Preface-Footnote-356627
+Node: History56773
+Node: Names59387
+Ref: Names-Footnote-160541
+Node: This Manual60704
+Ref: This Manual-Footnote-167634
+Node: Conventions67742
+Node: Manual History70171
+Ref: Manual History-Footnote-173196
+Ref: Manual History-Footnote-273237
+Node: How To Contribute73311
+Node: Acknowledgments74257
+Node: Getting Started79246
+Node: Running gawk81773
+Node: One-shot82991
+Node: Read Terminal84290
+Node: Long86344
+Node: Executable Scripts87994
+Ref: Executable Scripts-Footnote-190764
+Node: Comments90871
+Node: Quoting93401
+Node: DOS Quoting99045
+Node: Sample Data Files101125
+Node: Very Simple103760
+Node: Two Rules110014
+Node: More Complex111966
+Node: Statements/Lines114386
+Ref: Statements/Lines-Footnote-1119898
+Node: Other Features120187
+Node: When121155
+Ref: When-Footnote-1122957
+Node: Intro Summary123022
+Node: Invoking Gawk123976
+Node: Command Line125538
+Node: Options126388
+Ref: Options-Footnote-1145730
+Ref: Options-Footnote-2145971
+Node: Other Arguments145996
+Node: Naming Standard Input150161
+Node: Environment Variables151431
+Node: AWKPATH Variable152005
+Ref: AWKPATH Variable-Footnote-1155589
+Ref: AWKPATH Variable-Footnote-2155623
+Node: AWKLIBPATH Variable156014
+Ref: AWKLIBPATH Variable-Footnote-1157789
+Node: Other Environment Variables158184
+Node: Exit Status162660
+Node: Include Files163373
+Node: Loading Shared Libraries167618
+Node: Obsolete169110
+Node: Undocumented169858
+Node: Invoking Summary170155
+Node: Regexp173180
+Node: Regexp Usage174674
+Node: Escape Sequences176775
+Ref: Escape Sequences-Footnote-1184528
+Node: Regexp Operators184606
+Node: Regexp Operator Details185099
+Ref: Regexp Operator Details-Footnote-1193110
+Node: Interval Expressions193269
+Ref: Interval Expressions-Footnote-1195536
+Node: Bracket Expressions195634
+Ref: table-char-classes198383
+Ref: Bracket Expressions-Footnote-1201917
+Node: Leftmost Longest202186
+Node: Computed Regexps203542
+Node: GNU Regexp Operators207057
+Node: Case-sensitivity212154
+Ref: Case-sensitivity-Footnote-1215104
+Ref: Case-sensitivity-Footnote-2215347
+Node: Regexp Summary215459
+Node: Reading Files216981
+Node: Records219300
+Node: awk split records220575
+Node: gawk split records225457
+Ref: gawk split records-Footnote-1230745
+Node: Fields230782
+Ref: Fields-Footnote-1233698
+Node: Nonconstant Fields233818
+Ref: Nonconstant Fields-Footnote-1236290
+Node: Changing Fields236506
+Node: Field Separators242804
+Node: Default Field Splitting245673
+Node: Regexp Field Splitting246815
+Node: Single Character Fields250644
+Node: Comma Separated Fields251733
+Ref: table-csv-examples253143
+Node: Command Line Field Separator255448
+Node: Full Line Fields258824
+Ref: Full Line Fields-Footnote-1260402
+Ref: Full Line Fields-Footnote-2260448
+Node: Field Splitting Summary260553
+Node: Constant Size262870
+Node: Fixed width data263614
+Node: Skipping intervening267129
+Node: Allowing trailing data267931
+Node: Fields with fixed data268992
+Node: Splitting By Content270614
+Ref: Splitting By Content-Footnote-1276115
+Node: FS versus FPAT276280
+Node: Testing field creation277463
+Node: Multiple Line279237
+Node: Getline285706
+Node: Plain Getline288294
+Node: Getline/Variable290942
+Node: Getline/File292138
+Node: Getline/Variable/File293586
+Ref: Getline/Variable/File-Footnote-1295231
+Node: Getline/Pipe295327
+Node: Getline/Variable/Pipe298135
+Node: Getline/Coprocess299318
+Node: Getline/Variable/Coprocess300641
+Node: Getline Notes301407
+Node: Getline Summary305384
+Ref: table-getline-variants305828
+Node: Read Timeout306732
+Ref: Read Timeout-Footnote-1310694
+Node: Retrying Input310752
+Node: Command-line directories312019
+Node: Input Summary312957
+Node: Input Exercises316337
+Node: Printing316775
+Node: Print318718
+Node: Print Examples320219
+Node: Output Separators323064
+Node: OFMT325171
+Node: Printf326884
+Node: Basic Printf327689
+Node: Control Letters329324
+Node: Format Modifiers334770
+Node: Printf Examples341042
+Node: Redirection343583
+Node: Special FD351355
+Ref: Special FD-Footnote-1354653
+Node: Special Files354731
+Node: Other Inherited Files355360
+Node: Special Network356425
+Node: Special Caveats357313
+Node: Close Files And Pipes358296
+Ref: Close Files And Pipes-Footnote-1364420
+Node: Close Return Value364568
+Ref: table-close-pipe-return-values365839
+Ref: Close Return Value-Footnote-1366670
+Node: Noflush366826
+Node: Nonfatal368334
+Node: Output Summary370749
+Node: Output Exercises372035
+Node: Expressions372726
+Node: Values373926
+Node: Constants374604
+Node: Scalar Constants375299
+Ref: Scalar Constants-Footnote-1377875
+Ref: Scalar Constants-Footnote-2378125
+Node: Nondecimal-numbers378205
+Node: Regexp Constants381318
+Node: Using Constant Regexps381864
+Node: Standard Regexp Constants382510
+Node: Strong Regexp Constants385806
+Node: Variables389649
+Node: Using Variables390314
+Node: Assignment Options392288
+Node: Conversion394839
+Node: Strings And Numbers395371
+Ref: Strings And Numbers-Footnote-1398581
+Node: Locale influences conversions398690
+Ref: table-locale-affects401528
+Node: All Operators402170
+Node: Arithmetic Ops402811
+Node: Concatenation405634
+Ref: Concatenation-Footnote-1408570
+Node: Assignment Ops408689
+Ref: table-assign-ops413816
+Node: Increment Ops415197
+Node: Truth Values and Conditions418788
+Node: Truth Values419882
+Node: Typing and Comparison420962
+Node: Variable Typing421794
+Ref: Variable Typing-Footnote-1428438
+Ref: Variable Typing-Footnote-2428518
+Node: Comparison Operators428599
+Ref: table-relational-ops429026
+Node: POSIX String Comparison432702
+Ref: POSIX String Comparison-Footnote-1434459
+Ref: POSIX String Comparison-Footnote-2434602
+Node: Boolean Ops434687
+Ref: Boolean Ops-Footnote-1439361
+Node: Conditional Exp439457
+Node: Function Calls441237
+Node: Precedence445184
+Node: Locales449047
+Node: Expressions Summary450723
+Node: Patterns and Actions453378
+Node: Pattern Overview454514
+Node: Regexp Patterns456239
+Node: Expression Patterns456785
+Node: Ranges460690
+Node: BEGIN/END463864
+Node: Using BEGIN/END464673
+Ref: Using BEGIN/END-Footnote-1467581
+Node: I/O And BEGIN/END467691
+Node: BEGINFILE/ENDFILE470173
+Node: Empty473603
+Node: Using Shell Variables473920
+Node: Action Overview476256
+Node: Statements478692
+Node: If Statement480588
+Node: While Statement482151
+Node: Do Statement484239
+Node: For Statement485423
+Node: Switch Statement488778
+Node: Break Statement491327
+Node: Continue Statement493519
+Node: Next Statement495450
+Node: Nextfile Statement497929
+Node: Exit Statement500783
+Node: Built-in Variables503310
+Node: User-modified504487
+Node: Auto-set512694
+Ref: Auto-set-Footnote-1530778
+Ref: Auto-set-Footnote-2530996
+Node: ARGC and ARGV531052
+Node: Pattern Action Summary535481
+Node: Arrays538087
+Node: Array Basics539460
+Node: Array Intro540308
+Ref: figure-array-elements542319
+Ref: Array Intro-Footnote-1545172
+Node: Reference to Elements545304
+Node: Assigning Elements547824
+Node: Array Example548319
+Node: Scanning an Array550281
+Node: Controlling Scanning553376
+Ref: Controlling Scanning-Footnote-1560011
+Node: Numeric Array Subscripts560335
+Node: Uninitialized Subscripts562603
+Node: Delete564276
+Ref: Delete-Footnote-1567089
+Node: Multidimensional567146
+Node: Multiscanning570349
+Node: Arrays of Arrays572016
+Node: Arrays Summary576320
+Node: Functions578507
+Node: Built-in579709
+Node: Calling Built-in580898
+Node: Boolean Functions582938
+Node: Numeric Functions583500
+Ref: Numeric Functions-Footnote-1587685
+Ref: Numeric Functions-Footnote-2588368
+Ref: Numeric Functions-Footnote-3588420
+Node: String Functions588696
+Ref: String Functions-Footnote-1615147
+Ref: String Functions-Footnote-2615279
+Ref: String Functions-Footnote-3615535
+Node: Gory Details615622
+Ref: table-sub-escapes617631
+Ref: table-sub-proposed619183
+Ref: table-posix-sub620657
+Ref: table-gensub-escapes622309
+Ref: Gory Details-Footnote-1623137
+Node: I/O Functions623291
+Ref: table-system-return-values629968
+Ref: I/O Functions-Footnote-1632130
+Ref: I/O Functions-Footnote-2632278
+Node: Time Functions632398
+Ref: Time Functions-Footnote-1644110
+Ref: Time Functions-Footnote-2644178
+Ref: Time Functions-Footnote-3644340
+Ref: Time Functions-Footnote-4644451
+Ref: Time Functions-Footnote-5644567
+Ref: Time Functions-Footnote-6644794
+Node: Bitwise Functions645072
+Ref: table-bitwise-ops645670
+Ref: Bitwise Functions-Footnote-1652231
+Ref: Bitwise Functions-Footnote-2652408
+Node: Type Functions652603
+Node: I18N Functions654774
+Node: User-defined656509
+Node: Definition Syntax657255
+Ref: Definition Syntax-Footnote-1663110
+Node: Function Example663185
+Ref: Function Example-Footnote-1666164
+Node: Function Calling666186
+Node: Calling A Function666778
+Node: Variable Scope667748
+Node: Pass By Value/Reference670802
+Node: Function Caveats673530
+Ref: Function Caveats-Footnote-1675621
+Node: Return Statement675741
+Node: Dynamic Typing678773
+Node: Dynamic Typing Awk679355
+Node: Dynamic Typing Gawk681493
+Node: Shadowed Variables684863
+Node: Indirect Calls687211
+Node: Functions Summary698352
+Node: Library Functions701318
+Ref: Library Functions-Footnote-1704866
+Ref: Library Functions-Footnote-2705011
+Node: Library Names705186
+Ref: Library Names-Footnote-1708957
+Ref: Library Names-Footnote-2709184
+Node: General Functions709278
+Node: Strtonum Function710548
+Node: Assert Function713630
+Node: Round Function717185
+Node: Cliff Random Function718757
+Node: Ordinal Functions719782
+Ref: Ordinal Functions-Footnote-1722885
+Ref: Ordinal Functions-Footnote-2723137
+Node: Join Function723351
+Ref: Join Function-Footnote-1725149
+Node: Getlocaltime Function725353
+Node: Readfile Function729127
+Node: Shell Quoting731156
+Node: Isnumeric Function732612
+Node: To CSV Function734048
+Node: Data File Management736140
+Node: Filetrans Function736772
+Node: Rewind Function741048
+Node: File Checking743019
+Ref: File Checking-Footnote-1744385
+Node: Empty Files744590
+Node: Ignoring Assigns746653
+Node: Getopt Function748227
+Ref: Getopt Function-Footnote-1764045
+Node: Passwd Functions764257
+Ref: Passwd Functions-Footnote-1773392
+Node: Group Functions773480
+Ref: Group Functions-Footnote-1781604
+Node: Walking Arrays781815
+Node: Library Functions Summary784861
+Node: Library Exercises786281
+Node: Sample Programs786766
+Node: Running Examples787548
+Node: Clones788300
+Node: Cut Program789568
+Node: Egrep Program799992
+Node: Id Program809291
+Node: Split Program819383
+Ref: Split Program-Footnote-1829596
+Node: Tee Program829781
+Node: Uniq Program832687
+Node: Wc Program840547
+Node: Bytes vs. Characters840942
+Node: Using extensions842542
+Node: wc program843320
+Node: Miscellaneous Programs848313
+Node: Dupword Program849538
+Node: Alarm Program851587
+Node: Translate Program856490
+Ref: Translate Program-Footnote-1861199
+Node: Labels Program861477
+Ref: Labels Program-Footnote-1864912
+Node: Word Sorting864996
+Node: History Sorting869170
+Node: Extract Program871443
+Node: Simple Sed879696
+Node: Igawk Program882906
+Ref: Igawk Program-Footnote-1898116
+Ref: Igawk Program-Footnote-2898322
+Ref: Igawk Program-Footnote-3898452
+Node: Anagram Program898579
+Node: Signature Program901665
+Node: Programs Summary902916
+Node: Programs Exercises904170
+Ref: Programs Exercises-Footnote-1908999
+Node: Advanced Features909085
+Node: Nondecimal Data911566
+Node: Boolean Typed Values913196
+Node: Array Sorting915153
+Node: Controlling Array Traversal915882
+Ref: Controlling Array Traversal-Footnote-1924385
+Node: Array Sorting Functions924507
+Ref: Array Sorting Functions-Footnote-1930604
+Node: Two-way I/O930812
+Ref: Two-way I/O-Footnote-1938783
+Ref: Two-way I/O-Footnote-2938974
+Node: TCP/IP Networking939056
+Node: Profiling942224
+Node: Persistent Memory951894
+Ref: Persistent Memory-Footnote-1961334
+Node: Extension Philosophy961465
+Node: Advanced Features Summary962992
+Node: Internationalization965258
+Node: I18N and L10N966960
+Node: Explaining gettext967655
+Ref: Explaining gettext-Footnote-1973791
+Ref: Explaining gettext-Footnote-2973984
+Node: Programmer i18n974149
+Ref: Programmer i18n-Footnote-1979261
+Node: Translator i18n979310
+Node: String Extraction980140
+Ref: String Extraction-Footnote-1981316
+Node: Printf Ordering981414
+Ref: Printf Ordering-Footnote-1984272
+Node: I18N Portability984340
+Ref: I18N Portability-Footnote-1986900
+Node: I18N Example986967
+Ref: I18N Example-Footnote-1990361
+Ref: I18N Example-Footnote-2990434
+Node: Gawk I18N990551
+Node: I18N Summary991205
+Node: Debugger992602
+Node: Debugging993622
+Node: Debugging Concepts994071
+Node: Debugging Terms995888
+Node: Awk Debugging998491
+Ref: Awk Debugging-Footnote-1999464
+Node: Sample Debugging Session999600
+Node: Debugger Invocation1000150
+Node: Finding The Bug1001775
+Node: List of Debugger Commands1008407
+Node: Breakpoint Control1009784
+Node: Debugger Execution Control1013606
+Node: Viewing And Changing Data1017080
+Node: Execution Stack1020814
+Node: Debugger Info1022495
+Node: Miscellaneous Debugger Commands1026790
+Node: Readline Support1032031
+Node: Limitations1032969
+Node: Debugging Summary1035593
+Node: Namespaces1036892
+Node: Global Namespace1038132
+Node: Qualified Names1039566
+Node: Default Namespace1040601
+Node: Changing The Namespace1041374
+Node: Naming Rules1043056
+Node: Internal Name Management1045011
+Node: Namespace Example1046081
+Node: Inclusion For Namespaces1048660
+Node: Namespace And Features1051103
+Node: Namespace Summary1052565
+Node: Arbitrary Precision Arithmetic1054078
+Node: Computer Arithmetic1055597
+Ref: table-numeric-ranges1059522
+Ref: table-floating-point-ranges1060019
+Ref: Computer Arithmetic-Footnote-11060677
+Node: Math Definitions1060734
+Ref: table-ieee-formats1063766
+Node: MPFR features1064339
+Node: MPFR On Parole1064792
+Ref: MPFR On Parole-Footnote-11065633
+Node: MPFR Intro1065792
+Node: FP Math Caution1067477
+Ref: FP Math Caution-Footnote-11068549
+Node: Inexactness of computations1068923
+Node: Inexact representation1069954
+Node: Comparing FP Values1071335
+Node: Errors accumulate1072593
+Node: Strange values1074058
+Ref: Strange values-Footnote-11076712
+Node: Getting Accuracy1076817
+Node: Try To Round1079554
+Node: Setting precision1080461
+Ref: table-predefined-precision-strings1081166
+Node: Setting the rounding mode1083050
+Ref: table-gawk-rounding-modes1083432
+Ref: Setting the rounding mode-Footnote-11087484
+Node: Arbitrary Precision Integers1087669
+Ref: Arbitrary Precision Integers-Footnote-11090879
+Node: Checking for MPFR1091033
+Node: POSIX Floating Point Problems1092523
+Ref: POSIX Floating Point Problems-Footnote-11097343
+Node: Floating point summary1097381
+Node: Dynamic Extensions1099637
+Node: Extension Intro1101234
+Node: Plugin License1102536
+Node: Extension Mechanism Outline1103349
+Ref: figure-load-extension1103800
+Ref: figure-register-new-function1105378
+Ref: figure-call-new-function1106487
+Node: Extension API Description1108602
+Node: Extension API Functions Introduction1110331
+Ref: table-api-std-headers1112225
+Node: General Data Types1116666
+Ref: General Data Types-Footnote-11125812
+Node: Memory Allocation Functions1126115
+Ref: Memory Allocation Functions-Footnote-11130830
+Node: Constructor Functions1130929
+Node: API Ownership of MPFR and GMP Values1134830
+Node: Registration Functions1136383
+Node: Extension Functions1137087
+Node: Exit Callback Functions1142661
+Node: Extension Version String1143975
+Node: Input Parsers1144670
+Node: Output Wrappers1159289
+Node: Two-way processors1164131
+Node: Printing Messages1166484
+Ref: Printing Messages-Footnote-11167695
+Node: Updating ERRNO1167848
+Node: Requesting Values1168647
+Ref: table-value-types-returned1169400
+Node: Accessing Parameters1171459
+Node: Symbol Table Access1172740
+Node: Symbol table by name1173252
+Ref: Symbol table by name-Footnote-11176453
+Node: Symbol table by cookie1176585
+Ref: Symbol table by cookie-Footnote-11180854
+Node: Cached values1180918
+Ref: Cached values-Footnote-11184550
+Node: Array Manipulation1184707
+Ref: Array Manipulation-Footnote-11185806
+Node: Array Data Types1185843
+Ref: Array Data Types-Footnote-11188661
+Node: Array Functions1188757
+Node: Flattening Arrays1193786
+Node: Creating Arrays1200834
+Node: Redirection API1205676
+Node: Extension API Variables1208693
+Node: Extension Versioning1209416
+Ref: gawk-api-version1209845
+Node: Extension GMP/MPFR Versioning1211632
+Node: Extension API Informational Variables1213336
+Node: Extension API Boilerplate1214589
+Node: Changes from API V11218719
+Node: Finding Extensions1220351
+Node: Extension Example1220926
+Node: Internal File Description1221748
+Node: Internal File Ops1226040
+Ref: Internal File Ops-Footnote-11237590
+Node: Using Internal File Ops1237738
+Ref: Using Internal File Ops-Footnote-11240171
+Node: Extension Samples1240449
+Node: Extension Sample File Functions1242018
+Node: Extension Sample Fnmatch1250143
+Node: Extension Sample Fork1251738
+Node: Extension Sample Inplace1253014
+Node: Extension Sample Ord1257116
+Node: Extension Sample Readdir1257992
+Ref: table-readdir-file-types1258781
+Node: Extension Sample Revout1260137
+Node: Extension Sample Rev2way1260734
+Node: Extension Sample Read write array1261486
+Node: Extension Sample Readfile1264760
+Node: Extension Sample Time1265891
+Node: Extension Sample API Tests1267891
+Node: gawkextlib1268399
- Node: Extension summary1271432
- Node: Extension Exercises1275280
- Node: Language History1276550
- Node: V7/SVR3.11278262
- Node: SVR41280612
- Node: POSIX1282144
- Node: BTL1283569
- Node: POSIX/GNU1284336
- Ref: Gawk Extension Functions1287726
- Node: Feature History1291150
- Node: Common Extensions1311243
- Node: Ranges and Locales1312718
- Ref: Ranges and Locales-Footnote-11317503
- Ref: Ranges and Locales-Footnote-21317530
- Ref: Ranges and Locales-Footnote-31317765
- Node: Contributors1317988
- Node: History summary1324340
- Node: Installation1325782
- Node: Gawk Distribution1326746
- Node: Getting1327238
- Node: Extracting1328237
- Node: Distribution contents1329943
- Node: Unix Installation1337833
- Node: Quick Installation1338653
- Node: Compiling with MPFR1341193
- Node: Shell Startup Files1341899
- Node: Additional Configuration Options1343056
- Node: Configuration Philosophy1345439
- Node: Compiling from Git1347939
- Node: Building the Documentation1348500
- Node: Non-Unix Installation1349912
- Node: PC Installation1350388
- Node: PC Binary Installation1351257
- Node: PC Compiling1352150
- Node: PC Using1353328
- Node: Cygwin1357044
- Node: MSYS1358297
- Node: OpenVMS Installation1358923
- Node: OpenVMS Compilation1359604
- Ref: OpenVMS Compilation-Footnote-11361087
- Node: OpenVMS Dynamic Extensions1361145
- Node: OpenVMS Installation Details1362781
- Node: OpenVMS Running1365212
- Node: OpenVMS GNV1369349
- Node: Bugs1370104
- Node: Bug definition1371024
- Node: Bug address1374626
- Node: Usenet1378195
- Node: Performance bugs1379408
- Node: Asking for help1382412
- Node: Maintainers1384399
- Node: Other Versions1385426
- Node: Installation summary1395494
- Node: Notes1396876
- Node: Compatibility Mode1397686
- Node: Additions1398508
- Node: Accessing The Source1399453
- Node: Adding Code1400986
- Node: New Ports1408097
- Node: Derived Files1412600
- Ref: Derived Files-Footnote-11418411
- Ref: Derived Files-Footnote-21418446
- Ref: Derived Files-Footnote-31419057
- Node: Future Extensions1419171
- Node: Implementation Limitations1419841
- Node: Extension Design1421083
- Node: Old Extension Problems1422243
- Ref: Old Extension Problems-Footnote-11423815
- Node: Extension New Mechanism Goals1423876
- Ref: Extension New Mechanism Goals-Footnote-11427346
- Node: Extension Other Design Decisions1427547
- Node: Extension Future Growth1429744
- Node: Notes summary1430364
- Node: Basic Concepts1431574
- Node: Basic High Level1432259
- Ref: figure-general-flow1432541
- Ref: figure-process-flow1433243
- Ref: Basic High Level-Footnote-11436613
- Node: Basic Data Typing1436802
- Node: Glossary1440210
- Node: Copying1473092
- Node: GNU Free Documentation License1510650
- Node: Index1535773
++Node: Extension summary1271604
++Node: Extension Exercises1275452
++Node: Language History1276722
++Node: V7/SVR3.11278434
++Node: SVR41280784
++Node: POSIX1282316
++Node: BTL1283741
++Node: POSIX/GNU1284508
++Ref: Gawk Extension Functions1287898
++Node: Feature History1291322
++Node: Common Extensions1311415
++Node: Ranges and Locales1312890
++Ref: Ranges and Locales-Footnote-11317675
++Ref: Ranges and Locales-Footnote-21317702
++Ref: Ranges and Locales-Footnote-31317937
++Node: Contributors1318160
++Node: History summary1324512
++Node: Installation1325954
++Node: Gawk Distribution1326918
++Node: Getting1327410
++Node: Extracting1328409
++Node: Distribution contents1330115
++Node: Unix Installation1338005
++Node: Quick Installation1338825
++Node: Compiling with MPFR1341365
++Node: Shell Startup Files1342071
++Node: Additional Configuration Options1343228
++Node: Configuration Philosophy1345611
++Node: Compiling from Git1348111
++Node: Building the Documentation1348672
++Node: Non-Unix Installation1350084
++Node: PC Installation1350560
++Node: PC Binary Installation1351429
++Node: PC Compiling1352322
++Node: PC Using1353500
++Node: Cygwin1357216
++Node: MSYS1358469
++Node: OpenVMS Installation1359095
++Node: OpenVMS Compilation1359776
++Ref: OpenVMS Compilation-Footnote-11361259
++Node: OpenVMS Dynamic Extensions1361317
++Node: OpenVMS Installation Details1362953
++Node: OpenVMS Running1365384
++Node: OpenVMS GNV1369521
++Node: Bugs1370276
++Node: Bug definition1371196
++Node: Bug address1374798
++Node: Usenet1378367
++Node: Performance bugs1379580
++Node: Asking for help1382584
++Node: Maintainers1384571
++Node: Other Versions1385598
++Node: Installation summary1395666
++Node: Notes1397048
++Node: Compatibility Mode1397858
++Node: Additions1398680
++Node: Accessing The Source1399625
++Node: Adding Code1401158
++Node: New Ports1408269
++Node: Derived Files1412772
++Ref: Derived Files-Footnote-11418583
++Ref: Derived Files-Footnote-21418618
++Ref: Derived Files-Footnote-31419229
++Node: Future Extensions1419343
++Node: Implementation Limitations1420013
++Node: Extension Design1421255
++Node: Old Extension Problems1422415
++Ref: Old Extension Problems-Footnote-11423987
++Node: Extension New Mechanism Goals1424048
++Ref: Extension New Mechanism Goals-Footnote-11427518
++Node: Extension Other Design Decisions1427719
++Node: Extension Future Growth1429916
++Node: Notes summary1430536
++Node: Basic Concepts1431746
++Node: Basic High Level1432431
++Ref: figure-general-flow1432713
++Ref: figure-process-flow1433415
++Ref: Basic High Level-Footnote-11436785
++Node: Basic Data Typing1436974
++Node: Glossary1440382
++Node: Copying1473264
++Node: GNU Free Documentation License1510822
++Node: Index1535945
End Tag Table
http://git.sv.gnu.org/cgit/gawk.git/commit/?id=cfdf8297dac3a00ecef2483ef7add869483b0e26
commit cfdf8297dac3a00ecef2483ef7add869483b0e26
Author: Arnold D. Robbins <arnold@skeeve.com>
Date: Mon Feb 10 10:32:37 2025 +0200
Doc updates.
diff --git a/doc/ChangeLog b/doc/ChangeLog
index 30ee9cd2..1fa85c42 100644
--- a/doc/ChangeLog
+++ b/doc/ChangeLog
@@ -1,3 +1,13 @@
+2025-02-10 Arnold D. Robbins <arnold@skeeve.com>
+
+ * gawk.texi: Add indexing on `@' for typed regexp constants.
+ Add a link to the gawkextlib doc on SourceForge. Thanks to
+ Manuel Collado <mcollado2011@gmail.com> for both updates.
+
+ Unrelated:
+
+ * pm-gawk.texi: Updated from Terence Kelly.
+
2025-02-01 Arnold D. Robbins <arnold@skeeve.com>
* gawk.1: Remove note that persistent memory is experimental.
diff --git a/doc/gawk.info b/doc/gawk.info
index 907532bc..a7630d9a 100644
--- a/doc/gawk.info
+++ b/doc/gawk.info
@@ -30198,8 +30198,11 @@ provides a number of âgawkâ extensions, including
one for processing XML
files. This is the evolution of the original âxgawkâ (XML âgawkâ)
project.
- There are a number of extensions. Some of the more interesting ones
-are:
+ There are a number of extensions. The list of available extensions
+as well as their on-line full documentation can be seen in the
+âgawkextlibâ web pages (https://gawkextlib.sourceforge.net/).
+
+ Some of the more interesting extensions are:
⢠âabortâ extension. It allows you to exit immediately from your
âawkâ program without running the âENDâ rules.
@@ -36454,6 +36457,8 @@ Index
(line 9)
* @ (at-sign), @namespace directive, BEGIN, BEGINFILE, END, ENDFILE and:
Changing The Namespace.
(line 37)
+* @ (at-sign), strongly typed regexp constant: Strong Regexp Constants.
+ (line 24)
* * (asterisk), * operator, as multiplication operator: Precedence.
(line 54)
* * (asterisk), * operator, as regexp operator: Regexp Operator Details.
@@ -36799,6 +36804,8 @@ Index
(line 9)
* at-sign (@), @namespace directive, BEGIN, BEGINFILE, END, ENDFILE and:
Changing The Namespace.
(line 37)
+* at-sign (@), strongly typed regexp constant: Strong Regexp Constants.
+ (line 24)
* atan2: Numeric Functions. (line 11)
* automatic displays, in debugger: Debugger Info. (line 24)
* awf (amazingly workable formatter) program: Glossary. (line 23)
@@ -37886,7 +37893,7 @@ Index
(line 100)
* exp: Numeric Functions. (line 18)
* expand utility: Very Simple. (line 96)
-* Expat XML parser library: gawkextlib. (line 40)
+* Expat XML parser library: gawkextlib. (line 43)
* exponent: Numeric Functions. (line 18)
* expressions: Expressions. (line 6)
* expressions, as patterns: Expression Patterns. (line 6)
@@ -38360,7 +38367,7 @@ Index
* gettimeofday() extension function: Extension Sample Time.
(line 12)
* Ghostbusters: Gory Details. (line 6)
-* git utility: gawkextlib. (line 31)
+* git utility: gawkextlib. (line 34)
* git utility <1>: Other Versions. (line 18)
* git utility <2>: Accessing The Source.
(line 10)
@@ -39263,7 +39270,7 @@ Index
* Rankin, Pat: Acknowledgments. (line 60)
* Rankin, Pat <1>: Assignment Ops. (line 99)
* Rankin, Pat <2>: Contributors. (line 38)
-* RapidJson JSON parser library: gawkextlib. (line 37)
+* RapidJson JSON parser library: gawkextlib. (line 40)
* reada() extension function: Extension Sample Read write array.
(line 18)
* readable data files, checking: File Checking. (line 6)
@@ -40560,89 +40567,89 @@ Node: Extension Sample Readfile1261560
Node: Extension Sample Time1262691
Node: Extension Sample API Tests1264691
Node: gawkextlib1265199
-Node: Extension summary1268232
-Node: Extension Exercises1272080
-Node: Language History1273350
-Node: V7/SVR3.11275062
-Node: SVR41277412
-Node: POSIX1278944
-Node: BTL1280369
-Node: POSIX/GNU1281136
-Ref: Gawk Extension Functions1284526
-Node: Feature History1287950
-Node: Common Extensions1307852
-Node: Ranges and Locales1309327
-Ref: Ranges and Locales-Footnote-11314112
-Ref: Ranges and Locales-Footnote-21314139
-Ref: Ranges and Locales-Footnote-31314374
-Node: Contributors1314597
-Node: History summary1320949
-Node: Installation1322391
-Node: Gawk Distribution1323355
-Node: Getting1323847
-Node: Extracting1324846
-Node: Distribution contents1326552
-Node: Unix Installation1334442
-Node: Quick Installation1335262
-Node: Compiling with MPFR1337802
-Node: Shell Startup Files1338508
-Node: Additional Configuration Options1339665
-Node: Configuration Philosophy1342048
-Node: Compiling from Git1344548
-Node: Building the Documentation1345109
-Node: Non-Unix Installation1346521
-Node: PC Installation1346997
-Node: PC Binary Installation1347866
-Node: PC Compiling1348759
-Node: PC Using1349937
-Node: Cygwin1353653
-Node: MSYS1354906
-Node: OpenVMS Installation1355532
-Node: OpenVMS Compilation1356213
-Ref: OpenVMS Compilation-Footnote-11357696
-Node: OpenVMS Dynamic Extensions1357754
-Node: OpenVMS Installation Details1359390
-Node: OpenVMS Running1361821
-Node: OpenVMS GNV1365958
-Node: Bugs1366713
-Node: Bug definition1367633
-Node: Bug address1371235
-Node: Usenet1374804
-Node: Performance bugs1376017
-Node: Asking for help1379021
-Node: Maintainers1381008
-Node: Other Versions1382035
-Node: Installation summary1392103
-Node: Notes1393485
-Node: Compatibility Mode1394295
-Node: Additions1395117
-Node: Accessing The Source1396062
-Node: Adding Code1397595
-Node: New Ports1404706
-Node: Derived Files1409209
-Ref: Derived Files-Footnote-11415020
-Ref: Derived Files-Footnote-21415055
-Ref: Derived Files-Footnote-31415666
-Node: Future Extensions1415780
-Node: Implementation Limitations1416450
-Node: Extension Design1417692
-Node: Old Extension Problems1418852
-Ref: Old Extension Problems-Footnote-11420424
-Node: Extension New Mechanism Goals1420485
-Ref: Extension New Mechanism Goals-Footnote-11423955
-Node: Extension Other Design Decisions1424156
-Node: Extension Future Growth1426353
-Node: Notes summary1426973
-Node: Basic Concepts1428183
-Node: Basic High Level1428868
-Ref: figure-general-flow1429150
-Ref: figure-process-flow1429852
-Ref: Basic High Level-Footnote-11433222
-Node: Basic Data Typing1433411
-Node: Glossary1436819
-Node: Copying1469701
-Node: GNU Free Documentation License1507259
-Node: Index1532382
+Node: Extension summary1268404
+Node: Extension Exercises1272252
+Node: Language History1273522
+Node: V7/SVR3.11275234
+Node: SVR41277584
+Node: POSIX1279116
+Node: BTL1280541
+Node: POSIX/GNU1281308
+Ref: Gawk Extension Functions1284698
+Node: Feature History1288122
+Node: Common Extensions1308024
+Node: Ranges and Locales1309499
+Ref: Ranges and Locales-Footnote-11314284
+Ref: Ranges and Locales-Footnote-21314311
+Ref: Ranges and Locales-Footnote-31314546
+Node: Contributors1314769
+Node: History summary1321121
+Node: Installation1322563
+Node: Gawk Distribution1323527
+Node: Getting1324019
+Node: Extracting1325018
+Node: Distribution contents1326724
+Node: Unix Installation1334614
+Node: Quick Installation1335434
+Node: Compiling with MPFR1337974
+Node: Shell Startup Files1338680
+Node: Additional Configuration Options1339837
+Node: Configuration Philosophy1342220
+Node: Compiling from Git1344720
+Node: Building the Documentation1345281
+Node: Non-Unix Installation1346693
+Node: PC Installation1347169
+Node: PC Binary Installation1348038
+Node: PC Compiling1348931
+Node: PC Using1350109
+Node: Cygwin1353825
+Node: MSYS1355078
+Node: OpenVMS Installation1355704
+Node: OpenVMS Compilation1356385
+Ref: OpenVMS Compilation-Footnote-11357868
+Node: OpenVMS Dynamic Extensions1357926
+Node: OpenVMS Installation Details1359562
+Node: OpenVMS Running1361993
+Node: OpenVMS GNV1366130
+Node: Bugs1366885
+Node: Bug definition1367805
+Node: Bug address1371407
+Node: Usenet1374976
+Node: Performance bugs1376189
+Node: Asking for help1379193
+Node: Maintainers1381180
+Node: Other Versions1382207
+Node: Installation summary1392275
+Node: Notes1393657
+Node: Compatibility Mode1394467
+Node: Additions1395289
+Node: Accessing The Source1396234
+Node: Adding Code1397767
+Node: New Ports1404878
+Node: Derived Files1409381
+Ref: Derived Files-Footnote-11415192
+Ref: Derived Files-Footnote-21415227
+Ref: Derived Files-Footnote-31415838
+Node: Future Extensions1415952
+Node: Implementation Limitations1416622
+Node: Extension Design1417864
+Node: Old Extension Problems1419024
+Ref: Old Extension Problems-Footnote-11420596
+Node: Extension New Mechanism Goals1420657
+Ref: Extension New Mechanism Goals-Footnote-11424127
+Node: Extension Other Design Decisions1424328
+Node: Extension Future Growth1426525
+Node: Notes summary1427145
+Node: Basic Concepts1428355
+Node: Basic High Level1429040
+Ref: figure-general-flow1429322
+Ref: figure-process-flow1430024
+Ref: Basic High Level-Footnote-11433394
+Node: Basic Data Typing1433583
+Node: Glossary1436991
+Node: Copying1469873
+Node: GNU Free Documentation License1507431
+Node: Index1532554
End Tag Table
diff --git a/doc/gawk.texi b/doc/gawk.texi
index 34c7a837..344a4edd 100644
--- a/doc/gawk.texi
+++ b/doc/gawk.texi
@@ -11752,6 +11752,8 @@ are @dfn{strongly typed}; in other words, that denote a
regexp useful
for matching, and not an expression.
@cindex values @subentry regexp
+@cindex @code{@@} (at-sign) @subentry strongly typed regexp constant
+@cindex at-sign (@code{@@}) @subentry strongly typed regexp constant
@command{gawk} provides this feature. A strongly typed regexp constant
looks almost like a regular regexp constant, except that it is preceded
by an @samp{@@} sign:
@@ -40051,7 +40053,11 @@ project provides a number of @command{gawk}
extensions, including one for
processing XML files. This is the evolution of the original @command{xgawk}
(XML @command{gawk}) project.
-There are a number of extensions. Some of the more interesting ones are:
+There are a number of extensions. The list of available extensions as well
+as their on-line full documentation can be seen in the
+@uref{https://gawkextlib.sourceforge.net/, @code{gawkextlib} web pages}.
+
+Some of the more interesting extensions are:
@itemize @value{BULLET}
@item
diff --git a/doc/pm-gawk.info b/doc/pm-gawk.info
index c08c5b46..742f807b 100644
--- a/doc/pm-gawk.info
+++ b/doc/pm-gawk.info
@@ -1,7 +1,7 @@
This is pm-gawk.info, produced by makeinfo version 7.1 from
pm-gawk.texi.
-Copyright © 2022 Terence Kelly
+Copyright © 2022, 2025 Terence Kelly
<tpkelly@eecs.umich.edu>
<tpkelly@cs.princeton.edu>
<tpkelly@acm.org>
@@ -25,14 +25,14 @@ File: pm-gawk.info, Node: Top, Next: Introduction, Up:
(dir)
General Introduction
********************
-âgawkâ 5.2 introduces a _persistent memory_ feature that can "remember"
+âgawkâ 5.2 introduced a _persistent memory_ feature that can "remember"
script-defined variables and functions across executions; pass variables
between unrelated scripts without serializing/parsing text files; and
handle data sets larger than available memory plus swap. This
supplementary manual provides an in-depth look at persistent-memory
âgawkâ.
-Copyright © 2022 Terence Kelly
+Copyright © 2022, 2025 Terence Kelly
<tpkelly@eecs.umich.edu>
<tpkelly@cs.princeton.edu>
<tpkelly@acm.org>
@@ -65,7 +65,7 @@ File: pm-gawk.info, Node: Introduction, Next: Quick Start,
Prev: Top, Up: To
**************
-GNU AWK (âgawkâ) 5.2, expected in September 2022, introduces a new
+GNU AWK (âgawkâ) 5.2, released in September 2022, introduced a new
_persistent memory_ feature that makes AWK scripting easier and
sometimes improves performance. The new feature, called "pm-âgawkâ,"
can "remember" script-defined variables and functions across executions
@@ -73,7 +73,7 @@ and can pass variables and functions between unrelated
scripts without
serializing/parsing text files--all with near-zero fuss. pm-âgawkâ does
_not_ require non-volatile memory hardware nor any other exotic
infrastructure; it runs on the ordinary conventional computers and
-operating systems that most of us have been using for decades.
+operating systems that we've all been using for decades.
The main âgawkâ documentation(1) covers the basics of the new
@@ -126,11 +126,11 @@ File: pm-gawk.info, Node: Quick Start, Next: Examples,
Prev: Introduction, U
*************
Here's pm-âgawkâ in action at the âbashâ shell prompt (â$â):
- $ truncate -s 4096000 heap.pma
- $ export GAWK_PERSIST_FILE=heap.pma
- $ gawk 'BEGIN{myvar = 47}'
- $ gawk 'BEGIN{myvar += 7; print myvar}'
- 54
+ $ truncate -s 4096000 heap.pma
+ $ export GAWK_PERSIST_FILE=heap.pma
+ $ gawk 'BEGIN{myvar = 47}'
+ $ gawk 'BEGIN{myvar += 7; print myvar}'
+ 54 # '7' => not pm-gawk, crash => bad build
First, âtruncateâ creates an empty (all-zero-bytes) âheap fileâ where
pm-âgawkâ will store script variables; its size is a multiple of the
system page size (4 KiB). Next, âexportâ sets an environment variable
@@ -141,19 +141,21 @@ in the heap file and thereby outlive the interpreter
process that
initialized it. Finally, the fourth command invokes pm-âgawkâ on a
_different_ one-line script that increments and prints âmyvarâ. The
output shows that pm-âgawkâ has indeed "remembered" âmyvarâ across
-executions of unrelated scripts. (If the âgawkâ executable in your
-search â$PATHâ lacks the persistence feature, the output in the above
-example will be â7â instead of â54â. *Note Installation::.) To
disable
-persistence until you want it again, prevent âgawkâ from finding the
-heap file via âunset GAWK_PERSIST_FILEâ. To permanently "forget" script
-variables, delete the heap file.
-
+executions of unrelated scripts. To disable persistence until you want
+it again, prevent âgawkâ from finding the heap file via âunset
+GAWK_PERSIST_FILEâ. To permanently "forget" script variables, delete
+the heap file.
+ *Note Installation:: for two common problems and their fixes: If you
+run the example above and pm-âgawkâ crashes on the _second_ invocation,
+it is likely that your pm-âgawkâ was incorrectly built. If the printed
+output is â7â instead of â54â, the âgawkâ executable in your search
+â$PATHâ lacks the persistence feature.
Toggling persistence by âexportâ-ing and âunsetâ-ing "ambient"
envars
requires care: Forgetting to âunsetâ when you no longer want persistence
-can cause confusing bugs. Fortunately, âbashâ allows you to pass envars
-more deliberately, on a per-command basis:
+can cause surprises. Fortunately, âbashâ allows you to pass envars more
+deliberately, on a per-command basis:
$ rm heap.pma # start fresh
$ unset GAWK_PERSIST_FILE # eliminate ambient envar
$ truncate -s 4096000 heap.pma # create new heap file
@@ -171,8 +173,8 @@ does see the special envar and therefore uses the script
variable from
the heap file.
While sometimes less error prone than ambient envars, per-command
-envar passing as shown above is verbose and shouty. A shell alias saves
-keystrokes and reduces visual clutter:
+envar passing is verbose and shouty. A shell alias saves keystrokes and
+reduces visual clutter:
$ alias pm='GAWK_PERSIST_FILE=heap.pma'
$ pm gawk 'BEGIN{print ++myvar}'
55
@@ -224,10 +226,10 @@ parsing:
$ gawk 'BEGIN{print ts["river"], hf["river"]}'
26 142
- By making AWK more interactive, pm-âgawkâ invites casual
-conversations with data. If we're curious what words in âFinnâ are
-absent from âSawyerâ, answers (including "flapdoodle," "yellocution,"
-and "sockdolager") are easy to find:
+ pm-âgawkâ feels like it has a read-eval-print loop, which invites
+casual interactive conversations with data. If we're curious what words
+from âFinnâ are not in âSawyerâ, answers (including "flapdoodle,"
+"yellocution," and "sockdolager") are a few keystrokes away:
$ gawk 'BEGIN{for(w in hf) if (!(w in ts)) print w}'
Rumors of Twain's death may be exaggerated. If he publishes new
@@ -300,9 +302,9 @@ the number in the middle of a sorted list; if the length of
the list is
even, average the two numbers at the middle. The modes are the values
that occur most frequently.)
- Our third and final set of examples shows that pm-âgawkâ allows us to
-bundle both script-defined data and also user-defined _functions_ in a
-persistent heap that may be passed freely between unrelated AWK scripts.
+ Our final examples show that pm-âgawkâ allows us to bundle both
+script-defined data and also user-defined _functions_ in a persistent
+heap that we can pass freely between unrelated AWK scripts.
The following shell transcript repeatedly invokes pm-âgawkâ to create
and then employ a user-defined function. These separate invocations
@@ -311,7 +313,7 @@ file. Each invocation can add user-defined functions and
add or remove
data from the heap that subsequent invocations will access.
$ truncate -s 10M funcs.pma
$ export GAWK_PERSIST_FILE=funcs.pma
- $ gawk 'function count(A,t) {for(i in A)t++; return ""==t?0:t}'
+ $ gawk 'function count(A,t) {for(i in A)t++; return t+0}'
$ gawk 'BEGIN { a["x"] = 4; a["y"] = 5; a["z"] = 6 }'
$ gawk 'BEGIN { print count(a) }'
3
@@ -339,7 +341,7 @@ count them.
collection of user-defined functions that perform basic operations on
quadratic polynomials: evaluation at a given point, computing the
discriminant, and using the quadratic formula to find the roots. It
-then factorizes x^2 + x - 12 into (x - 3)(x + 4).
+then factors x^2 + x - 12 into (x - 3)(x + 4).
#!/bin/sh
rm -f poly.pma
truncate -s 10M poly.pma
@@ -499,8 +501,8 @@ Changing these parameters can prevent wasteful eager
paging:(1)
$ echo 300000 | sudo tee /proc/sys/vm/dirty_expire_centisecs
$ echo 50000 | sudo tee /proc/sys/vm/dirty_writeback_centisecs
Tuning paging parameters can help non-persistent âgawkâ as well as
-pm-âgawkâ. [Disclaimer: OS tuning is an occult art, and your mileage
-may vary.]
+pm-âgawkâ.
+[*Disclaimer*: OS tuning is an occult art, and your mileage may vary.]
---------- Footnotes ----------
@@ -536,10 +538,10 @@ to create a logically enormous sparse file and check its
physical size:
$ du -h sparse
0 sparse
Whereas âlsâ reports the logical file size that we expect (one TiB or 2
-raised to the power 40 bytes), âduâ reveals that the file occupies no
-storage whatsoever. The file system will allocate physical storage
-resources beneath this file as data is written to it; reading unwritten
-regions of the file yields zeros.
+raised to the power 40 bytes), âduâ reveals that the file consumes
+_zero_ storage. The file system will allocate physical storage beneath
+this file as data are written to it; reading unwritten regions yields
+zeros.
The "pay as you go" storage cost of sparse files offers both
convenience and control for pm-âgawkâ users. If your file system
@@ -552,23 +554,26 @@ initial size to whatever bound you wish to enforce; it
won't eat more
disk than that. Copying sparse files with GNU âcpâ creates sparse
copies by default.
- File-system encryption can preclude sparse files: If the cleartext of
+ To maximize storage frugality we sometimes want to "re-sparsify" heap
+files cluttered with de-allocated memory that pm-âgawkâ no longer needs.
+A stand-alone utility, âpma_samâ, is provided for this purpose at the
+âpmaâ web site.
+
+ File-system encryption can preclude sparse files: If the plaintext of
a byte offset range within a file is all zero bytes, the corresponding
-ciphertext probably shouldn't be all zeros! Encrypting at the storage
-layer instead of the file system layer may offer acceptable security
-while still permitting file systems to implement sparse files.
-
- Sometimes you might prefer a dense heap file backed by pre-allocated
-storage resources, for example to increase the likelihood that
-pm-âgawkâ's internal memory allocation will succeed until the persistent
-heap occupies the entire heap file. The âfallocateâ utility will do the
-trick:
+ciphertext mustn't be all zeros! Encrypting at the storage layer
+instead of the file system may offer acceptable security while still
+permitting sparse files.
+
+ Sometimes you want a dense heap file backed by pre-allocated storage,
+e.g., to ensure that pm-âgawkâ's internal memory allocation will succeed
+until the persistent heap fills the entire file. The âfallocateâ
+utility does the trick:
$ fallocate -l 1M mibi
$ ls -l mibi
-rw-rw-r--. 1 me me 1048576 Aug 5 23:18 mibi
$ du -h mibi
- 1.0M mibi
-We get the MiB we asked for, both logically and physically.
+ 1.0M mibi # We get our MiB, both logically & physically.
File: pm-gawk.info, Node: Persistence versus Durability, Next: Experiments,
Prev: Sparse Heap Files, Up: Performance
@@ -651,8 +656,8 @@ File: pm-gawk.info, Node: Experiments, Next: Results,
Prev: Persistence versu
The C-shell (âcshâ) script listed below illustrates concepts and
implements tips presented in this chapter. It produced the results
discussed in *note Results:: in roughly 20 minutes on an aging laptop.
-You can cut and paste the code listing below into a file, or download it
-from <http://web.eecs.umich.edu/~tpkelly/pma/>.
+To reproduce my experiments, cut/paste the listing below into a file;
+take care that no lines are duplicated or omitted.
The script measures the performance of four different ways to support
word frequency queries over a text corpus: The naïve approach of reading
@@ -856,7 +861,7 @@ footprint is very small while serving a query that accesses
a single
array element. The upside of the large heap file is O(1) access instead
of O(W)--a classic time-space tradeoff. If storage is a scarce
resource, all three intermediate files can be compressed, âfreqtblâ by a
-factor of roughly 2.7, ârwarrayâ by roughly 5.6x, and pm-âgawkâ by
+factor of roughly 2.7x, ârwarrayâ by roughly 5.6x, and pm-âgawkâ by
roughly 11x using âxzâ. Compression is CPU-intensive and slow, another
time-space tradeoff.
@@ -952,7 +957,9 @@ suggested the min/max/mean example in *note Examples::, and
also the
exercise of making Kernighan & Pike's "Markov" script persistent. Volos
provided and tested the advice on tuning OS parameters in *note Virtual
Memory and Big Data::. Stan Park provided insights about virtual
-memory, file systems, and utilities.
+memory, file systems, and utilities. Antonio Giovanni Colombo
+translated an earlier edition of this manual into Italian in August
+2022; see file âdoc/it/pm-gawk.texiâ in the âgawkâ distribution
tarball.
File: pm-gawk.info, Node: Installation, Next: Debugging, Prev:
Acknowledgments, Up: Top
@@ -960,16 +967,99 @@ File: pm-gawk.info, Node: Installation, Next:
Debugging, Prev: Acknowledgment
Appendix A Installation
***********************
-âgawkâ 5.2 featuring persistent memory is expected to be released in
-September 2022; look for it at <http://ftp.gnu.org/gnu/gawk/>. If 5.2
-is not released yet, the master git branch is available at
-<http://git.savannah.gnu.org/cgit/gawk.git/snapshot/gawk-master.tar.gz>.
+Users may obtain pm-âgawkâ via software package management systems or
+via manual installation. Each approach has pros and cons.
+
+ *Packages* Delegating the chore of installation to software package
+management systems, such as those associated with major Linux
+distributions including Ubuntu and Fedora, is easier--in theory.
+Software packages on some Linux distributions, however, lag years behind
+the latest software releases. Therefore relatively new features might
+be available only in "upstream" releases but not "downstream" packages.
+For example, as of February 2025 the package-installed âgawkâ on Ubuntu
+Linux is pre-5.2 and therefore lacks the persistence feature introduced
+in âgawkâ 5.2, which was released way back in September 2022! If the
+first output line of âgawk --versionâ shows 5.2 or later and "âPMAâ,"
+you've got a pm-âgawkâ that supports persistence; otherwise see "Manual
+Install" below.
+
+ Another problem with software installed by package managers is that
+the software may have been compiled/built incorrectly. For example, as
+of February 2025 the package-installed âgawkâ 5.3.0 on Fedora 41 was
+incorrectly created as a position-independent executable (PIE), despite
+the official âgawkâ distribution's build system very deliberately and
+explicitly disabling PIE. The result, noted in *note Quick Start::, is
+that pm-âgawkâ crashes the second time it is invoked. The details of
+the Fedora package bug, and remedies that are works in progress as of
+early February 2025, are available at
+ <https://bugzilla.redhat.com/show_bug.cgi?id=2341653>
+The âfileâ utility reveals whether âgawkâ was incorrectly built. On
+Fedora 41, for example:
+ $ which gawk
+ /usr/bin/gawk
+ $ file /usr/bin/gawk
+ /usr/bin/gawk: ELF 64-bit LSB pie ...
+That little word "âpieâ" is likely to blame if your pm-âgawkâ crashes
on
+the _second_ invocation. By default, PIEs run with address-space layout
+randomization (ASLR), which âgawkâ 5.2 thru the current 5.3.1 do not
+tolerate.
+
+ (Note: The persistent memory allocator that enables persistent-memory
+âgawkâ--the âpmaâ library--is perfectly compatible with PIE and ASLR.
+The incompatibility of âgawkâ and ASLR arises from the interpreter's
+internal data structures, which require that function pointers be
+consistent across invocations. ASLR introduces gratuitous
+inconsistencies into these pointers.)
+
+ While we're waiting for an elegant and permanent fix for the Fedora
+41 PIE bug, we can use a klugey workaround: Run the pm-âgawkâ
+interpreter via âsetarch -Râ, which disables ASLR despite the PIE.
+Here's the first example from *note Quick Start::, first without the
+fix, then with the fix:
+ $ truncate -s 4096000 heap.pma
+ $ export GAWK_PERSIST_FILE=heap.pma
+ $ gawk 'BEGIN{myvar = 47}'
+ $ gawk 'BEGIN{myvar += 7; print myvar}'
+ Segmentation fault (core dumped) # thanks to PIE
+
+ $ rm -f heap.pma # discard heap from first try
+ $ truncate -s 4096000 heap.pma # start fresh
+ $ setarch -R gawk 'BEGIN{myvar = 47}'
+ $ setarch -R gawk 'BEGIN{myvar += 7; print myvar}'
+ 54 # success
+You can define a shell alias to expand the familiar and ergonomic âgawkâ
+into the cumbersome and verbose âsetarch -R gawkâ.
+
+ Alternatively, it might be possible to disable ASLR system-wide by
+using âsysctlâ to twiddle variables such as
âkernel.randomize_va_spaceâ.
+I have not investigated such measures, which should be used with
+caution. Fully understand the implications of such a blunt system-wide
+change before making it.
+
+ It's reasonable to expect that, in the normal course of events,
+correctly built pm-âgawkâ will eventually find its way into the default
+package-installed âgawkâ on major GNU/Linux distros. Meanwhile, manual
+installation is a fairly easy way to get a working pm-âgawkâ.
+
+ *Manual Install* Manually compiling âgawkâ from source code in the
+latest upstream release gives you the most recent stable version of
+âgawkâ, built in accordance with with the maintainer's recipe, with no
+meddling by intermediaries. Therefore a manual install won't suffer
+from bugs such as the PIE bug discussed above. Download the latest
+release here:
+ <https://ftp.gnu.org/gnu/gawk/>
+ <https://ftp.gnu.org/gnu/gawk/gawk-5.3.1.tar.xz>
+
+ Finally, for the ultimate in bleeding-edge upstream freshness,
+adventurous do-it-yourselfers can grab the âgitâ master branch from
+<http://git.savannah.gnu.org/cgit/gawk.git/snapshot/gawk-master.tar.gz>
Unpack the tarball, run â./bootstrap.shâ, â./configureâ, âmakeâ,
and
-âmake checkâ, then try some of the examples presented earlier. In the
-normal course of events, 5.2 and later âgawkâ releases featuring
-pm-âgawkâ will appear in the software package management systems of
-major GNU/Linux distros. Eventually pm-âgawkâ will be available in the
-default âgawkâ on such systems.
+âmake checkâ, then try some of the examples presented earlier.
+
+ As of February 2025, pm-âgawkâ is supported on most, but not all,
+major Unix-like platforms. The âgawkâ build system decides whether to
+include support for persistence; see the âgawkâ documentation and file
+â./m4/pma.m4â in the âgawkâ distribution.
File: pm-gawk.info, Node: Debugging, Next: History, Prev: Installation,
Up: Top
@@ -990,10 +1080,12 @@ variables such as âNRâ are always reset to zero
every time the
interpreter runs. See the discussion of initialization surrounding the
min/max/mean script in *note Examples::.
- If you suspect a persistence-related bug in pm-âgawkâ, you can set an
-environment variable that will cause its persistent heap module, âpmaâ,
-to emit more verbose error messages; for details see the main âgawkâ
-documentation.
+ If pm-âgawkâ crashes on the _second_ invocation that uses a
+particular heap file, see the discussion of PIE in *note Installation::.
+If you suspect some other kind of persistence-related bug in pm-âgawkâ,
+you can set an environment variable that will cause its persistent heap
+module, âpmaâ, to emit more verbose error messages; for details see the
+main âgawkâ documentation.
Programmers: You can re-compile âgawkâ with assertions enabled, which
will trigger extensive integrity checks within âpmaâ. Ensure that
@@ -1017,8 +1109,8 @@ allocator, âpmaâ, whose design is described in
trace the evolutionary paths that led to âpmaâ and pm-âgawkâ.
I wrote many AWK scripts during my dissertation research on Web
-caching twenty years ago, most of which processed log files from Web
-servers and Web caches. Persistent âgawkâ would have made these scripts
+caching 25 years ago, most of which processed log files from Web servers
+and Web caches. Persistent âgawkâ would have made these scripts
smaller, faster, and easier to write, but at the time I was unable even
to imagine that pm-âgawkâ is possible. So I wrote a lot of bothersome,
inefficient code that manually dumped and re-loaded AWK script variables
@@ -1049,7 +1141,7 @@ strengthens the semantics of ordinary standard
âmsync()â by guaranteeing
that the durable state of a memory-mapped file always reflects the most
recent successful âmsync()â call, even in the presence of failures such
as power outages and OS or application crashes. The original Linux
-kernel FAMS prototype is described in a paper by Park et al. in EuroSys
+kernel FAMS prototype is described in a paper by Park et al. in EuroSys
2013. My colleagues and I subsequently implemented FAMS in several
different ways including in file systems (FAST 2015) and user-space
libraries. My most recent FAMS implementation, which leverages the
@@ -1059,20 +1151,20 @@ ubiquitous GNU âdbmâ (âgdbmâ) database
(<https://queue.acm.org/detail.cfm?id=3487353>).
In recent years my attention has returned to the advantages of
-persistent memory programming, lately a hot topic thanks to the
-commercial availability of byte-addressable non-volatile memory hardware
-(which, confusingly, is nowadays marketed as "persistent memory"). The
-software abstraction of persistent memory and the corresponding
-programming style, however, are perfectly compatible with _conventional_
-computers--machines with neither non-volatile memory nor any other
-special hardware or software. I wrote a few papers making this point,
-for example <https://queue.acm.org/detail.cfm?id=3358957>.
+persistent memory programming, which was a hot topic circa COVID thanks
+to the fleeting commercial availability of byte-addressable non-volatile
+memory hardware (Intel Optane, confusingly marketed as "persistent
+memory"). The software abstraction of persistent memory and the
+corresponding programming style, however, are perfectly compatible with
+_conventional_ computers--machines with neither non-volatile memory nor
+any other special hardware or software. Several papers make this point,
+e.g., <https://queue.acm.org/detail.cfm?id=3358957>
In early 2022 I wrote a new stand-alone persistent memory allocator,
âpmaâ, to make persistent memory programming easy on conventional
hardware. The âpmaâ interface is compatible with âmalloc()â and,
unlike
Ken's allocator, âpmaâ is not coupled to a particular crash-tolerance
-mechanism. Using âpmaâ is easy and, at least to some, enjoyable.
+mechanism. Using âpmaâ is easy and fun.
Ken had been integrated into prototype forks of both the V8
JavaScript interpreter and a Scheme interpreter, so it was natural to
@@ -1094,14 +1186,12 @@ respects, however, the two implementations are similar.
A description
of the prototype, including performance measurements, is available at
<http://nvmw.ucsd.edu/nvmw2022-program/nvmw2022-data/nvmw2022-paper35-final_version_your_extended_abstract.pdf>.
-
-
I enjoy several aspects of pm-âgawkâ. It's unobtrusive; as you gain
familiarity and experience, it fades into the background of your
scripting. It's simple in both concept and implementation, and more
importantly it simplifies your scripts; much of its value is measured
not in the code it enables you to write but rather in the code it lets
-you discard. It's all that I needed for my dissertation research twenty
+you discard. It's all that I needed for my dissertation research 25
years ago, and more. Anecdotally, it appears to inspire creativity in
early adopters, who have devised uses that pm-âgawkâ's designers never
anticipated. I'm curious to see what new purposes you find for it.
@@ -1109,30 +1199,30 @@ anticipated. I'm curious to see what new purposes you
find for it.
Tag Table:
-Node: Top805
-Node: Introduction2014
-Ref: Introduction-Footnote-14423
-Node: Quick Start4527
-Node: Examples7430
-Node: Performance16514
-Node: Constant-Time Array Access17222
-Node: Virtual Memory and Big Data20572
-Ref: Virtual Memory and Big Data-Footnote-124175
-Node: Sparse Heap Files24315
-Node: Persistence versus Durability27389
-Ref: Persistence versus Durability-Footnote-130850
-Ref: Persistence versus Durability-Footnote-231096
-Node: Experiments31490
-Node: Results43225
-Node: Data Integrity46849
-Ref: Data Integrity-Footnote-149687
-Ref: Data Integrity-Footnote-249784
-Ref: Data Integrity-Footnote-349936
-Ref: Data Integrity-Footnote-450030
-Node: Acknowledgments50409
-Node: Installation51603
-Node: Debugging52433
-Node: History54174
+Node: Top811
+Node: Introduction2026
+Ref: Introduction-Footnote-14429
+Node: Quick Start4533
+Node: Examples7628
+Node: Performance16717
+Node: Constant-Time Array Access17425
+Node: Virtual Memory and Big Data20775
+Ref: Virtual Memory and Big Data-Footnote-124379
+Node: Sparse Heap Files24519
+Node: Persistence versus Durability27708
+Ref: Persistence versus Durability-Footnote-131169
+Ref: Persistence versus Durability-Footnote-231415
+Node: Experiments31809
+Node: Results43544
+Node: Data Integrity47169
+Ref: Data Integrity-Footnote-150007
+Ref: Data Integrity-Footnote-250104
+Ref: Data Integrity-Footnote-350256
+Ref: Data Integrity-Footnote-450350
+Node: Acknowledgments50729
+Node: Installation52098
+Node: Debugging57393
+Node: History59288
End Tag Table
diff --git a/doc/pm-gawk.texi b/doc/pm-gawk.texi
index 3f68099a..ff9cc73d 100644
--- a/doc/pm-gawk.texi
+++ b/doc/pm-gawk.texi
@@ -2,9 +2,11 @@
@c TODO: Checklist for release:
@c revise all U P D A T E items as appropriate
+@c check all URLs by clicking in PDF version
@c check all to-do notes
@c remove most comments
-@c spell check (last 2am 15 Aug 2022)
+@c spell check
+@c post final version to pma web site
@c verbatim limits: 47 rows x 75 cols, smallformat 58 x 90
@@ -42,9 +44,8 @@ pm-@gwk{}
@copying
@noindent
@c UPDATE copyright info below
-Copyright @copyright{} 2022 Terence Kelly @*
+Copyright @copyright{} 2022, 2025 Terence Kelly @*
@ifnottex
-@noindent
@email{tpkelly@@eecs.umich.edu} @*
@email{tpkelly@@cs.princeton.edu} @*
@email{tpkelly@@acm.org} @*
@@ -65,15 +66,16 @@ A copy of the license is available at @*
@titlepage
@title @value{TYTL}
@c UPDATE date below
-@subtitle 16 August 2022
-@subtitle @gwk{} version 5.2
-@subtitle @pmg{} version 2022.08Aug.03.1659520468 (Avon 7)
+@subtitle 9 February 2025
+@subtitle @gwk{} version 5.3.1
+@subtitle @pmg{} version 2022.10Oct.30.1667172241 (Avon 8)
@author Terence Kelly
@author @email{tpkelly@@eecs.umich.edu}
@author @email{tpkelly@@cs.princeton.edu}
@author @email{tpkelly@@acm.org}
@author @url{http://web.eecs.umich.edu/~tpkelly/pma/}
@author @url{https://dl.acm.org/profile/81100523747}
+@author @url{https://queue.acm.org/DrillBits}
@vskip 0pt plus 1filll
@insertcopying
@end titlepage
@@ -87,7 +89,7 @@ A copy of the license is available at @*
@ifnotxml
@ifnotdocbook
@top General Introduction
-@gwk{} 5.2 introduces a @emph{persistent memory} feature that can
+@gwk{} 5.2 introduced a @emph{persistent memory} feature that can
``remember'' script-defined variables and functions across executions;
pass variables between unrelated scripts without serializing/parsing
text files; and handle data sets larger than available memory plus
@@ -99,6 +101,9 @@ persistent-memory @gwk{}.
@end ifnotxml
@end ifnottex
+@c UPDATE: ensure that TOC below matches order of sections and
+@c homebrew TOC in Introduction
+
@menu
* Introduction::
* Quick Start::
@@ -117,8 +122,7 @@ persistent-memory @gwk{}.
@sp 1
-@c UPDATE below after official release
-GNU AWK (@gwk{}) 5.2, expected in September 2022, introduces a new
+GNU AWK (@gwk{}) 5.2, released in September 2022, introduced a new
@emph{persistent memory} feature that makes AWK scripting easier and
sometimes improves performance. The new feature, called ``@pmg{},''
can ``remember'' script-defined variables and functions across
@@ -126,8 +130,7 @@ executions and can pass variables and functions between
unrelated
scripts without serializing/parsing text files---all with near-zero
fuss. @pmg{} does @emph{not} require non-volatile memory hardware nor
any other exotic infrastructure; it runs on the ordinary conventional
-computers and operating systems that most of us have been using for
-decades.
+computers and operating systems that we've all been using for decades.
@sp 1
@@ -142,7 +145,7 @@ additional detail, tutorial examples, and a peek under the
hood of
@pmg{}. If you're familiar with @gwk{} and Unix-like environments,
dive straight in: @*
-@itemize @c @w{}
+@itemize
@item @ref{Quick Start} hits the ground running with a few keystrokes.
@item @ref{Examples} shows how @pmg{} streamlines typical AWK scripting.
@item @ref{Performance} covers asymptotic efficiency, OS tuning, and more.
@@ -163,6 +166,9 @@ You can find the latest version of this manual, and also the
allocator used in @pmg{}: @*
@center @url{http://web.eecs.umich.edu/~tpkelly/pma/}
+@c not citing this because it might not be as fresh as pma site
+@c https://www.gnu.org/software/gawk/manual/pm-gawk/
+
@sp 1
@noindent
@@ -194,11 +200,11 @@ Feel free to send me questions, suggestions, and
experiences: @*
Here's @pmg{} in action at the @command{bash} shell prompt (@samp{$}):
@verbatim
- $ truncate -s 4096000 heap.pma
- $ export GAWK_PERSIST_FILE=heap.pma
- $ gawk 'BEGIN{myvar = 47}'
- $ gawk 'BEGIN{myvar += 7; print myvar}'
- 54
+ $ truncate -s 4096000 heap.pma
+ $ export GAWK_PERSIST_FILE=heap.pma
+ $ gawk 'BEGIN{myvar = 47}'
+ $ gawk 'BEGIN{myvar += 7; print myvar}'
+ 54 # '7' => not pm-gawk, crash => bad build
@end verbatim
@noindent
First, @command{truncate} creates an empty (all-zero-bytes) @dfn{heap
@@ -212,18 +218,19 @@ the interpreter process that initialized it. Finally,
the fourth
command invokes @pmg{} on a @emph{different} one-line script that
increments and prints @code{myvar}. The output shows that @pmg{} has
indeed ``remembered'' @code{myvar} across executions of unrelated
-scripts. (If the @gwk{} executable in your search @env{$PATH} lacks
-the persistence feature, the output in the above example will be
-@samp{7} instead of @samp{54}. @xref{Installation}.) To disable
-persistence until you want it again, prevent @gwk{} from finding the
-heap file via @command{unset GAWK_PERSIST_FILE}. To permanently
-``forget'' script variables, delete the heap file.
+scripts. To disable persistence until you want it again, prevent
+@gwk{} from finding the heap file via @samp{unset GAWK_PERSIST_FILE}.
+To permanently ``forget'' script variables, delete the heap file.
-@sp 2
+@xref{Installation} for two common problems and their fixes: If you
+run the example above and @pmg{} crashes on the @emph{second}
+invocation, it is likely that your @pmg{} was incorrectly built. If
+the printed output is @samp{7} instead of @samp{54}, the @gwk{}
+executable in your search @env{$PATH} lacks the persistence feature.
Toggling persistence by @command{export}-ing and @command{unset}-ing
``ambient'' envars requires care: Forgetting to @command{unset} when
-you no longer want persistence can cause confusing bugs. Fortunately,
+you no longer want persistence can cause surprises. Fortunately,
@command{bash} allows you to pass envars more deliberately, on a
per-command basis:
@verbatim
@@ -246,8 +253,8 @@ invocation does see the special envar and therefore uses
the script
variable from the heap file.
While sometimes less error prone than ambient envars, per-command
-envar passing as shown above is verbose and shouty. A shell alias
-saves keystrokes and reduces visual clutter:
+envar passing is verbose and shouty. A shell alias saves keystrokes
+and reduces visual clutter:
@verbatim
$ alias pm='GAWK_PERSIST_FILE=heap.pma'
$ pm gawk 'BEGIN{print ++myvar}'
@@ -316,10 +323,11 @@ input parsing:
26 142
@end verbatim
-By making AWK more interactive, @pmg{} invites casual conversations
-with data. If we're curious what words in @cite{Finn} are absent from
-@cite{Sawyer}, answers (including ``flapdoodle,'' ``yellocution,'' and
-``sockdolager'') are easy to find:
+@pmg{} feels like it has a read-eval-print loop, which invites casual
+interactive conversations with data. If we're curious what words from
+@cite{Finn} are not in @cite{Sawyer}, answers (including
+``flapdoodle,'' ``yellocution,'' and ``sockdolager'') are a few
+keystrokes away:
@verbatim
$ gawk 'BEGIN{for(w in hf) if (!(w in ts)) print w}'
@end verbatim
@@ -423,23 +431,9 @@ that occur most frequently.)
@page
@c = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = =
-Our third and final set of examples shows that @pmg{} allows us to
-bundle both script-defined data and also user-defined @emph{functions}
-in a persistent heap that may be passed freely between unrelated AWK
-scripts.
-
-@c ADR doesn't like return in count() below
-@c TK: it was put there for a reason:
-@c $ truncate -s 10M funcs.pma
-@c $ export GAWK_PERSIST_FILE=funcs.pma
-@c $ gawk 'function count(A,t) {for(i in A)t++; return t}'
-@c $ gawk 'BEGIN { a["x"] = 4; a["y"] = 5; a["z"] = 6 }'
-@c $ gawk 'BEGIN { print count(a) }'
-@c 3
-@c $ gawk 'BEGIN { delete a }'
-@c $ gawk 'BEGIN { print count(a) }'
-@c [!!blank line, not zero!!]
-@c $
+Our final examples show that @pmg{} allows us to bundle both
+script-defined data and also user-defined @emph{functions} in a
+persistent heap that we can pass freely between unrelated AWK scripts.
The following shell transcript repeatedly invokes @pmg{} to create and
then employ a user-defined function. These separate invocations
@@ -450,7 +444,7 @@ remove data from the heap that subsequent invocations will
access.
@verbatim
$ truncate -s 10M funcs.pma
$ export GAWK_PERSIST_FILE=funcs.pma
- $ gawk 'function count(A,t) {for(i in A)t++; return ""==t?0:t}'
+ $ gawk 'function count(A,t) {for(i in A)t++; return t+0}'
$ gawk 'BEGIN { a["x"] = 4; a["y"] = 5; a["z"] = 6 }'
$ gawk 'BEGIN { print count(a) }'
3
@@ -477,14 +471,11 @@ reduced count, 2. The two commands after that delete the
entire array
and print a count of zero. Finally, the last two @pmg{} commands
populate the array with 47 entries and count them.
-@c I could be persuaded to leave the polynomial example as an
-@c exercise, offering to send my answer to readers upon request.
-
The following shell script invokes @pmg{} repeatedly to create a
collection of user-defined functions that perform basic operations on
quadratic polynomials: evaluation at a given point, computing the
discriminant, and using the quadratic formula to find the roots. It
-then factorizes @math{x^2 + x - 12} into @math{(x - 3)(x + 4)}.
+then factors @math{x^2 + x - 12} into @math{(x - 3)(x + 4)}.
@smallformat
@verbatim
#!/bin/sh
@@ -509,7 +500,6 @@ then factorizes @math{x^2 + x - 12} into @math{(x - 3)(x +
4)}.
rm -f poly.pma
@end verbatim
@end smallformat
-@noindent
@page
@c ==================================================================
@@ -557,6 +547,8 @@ implements arrays as hash tables.
@c so as the size of a corpus increases without bound, the ratio of
@c vocabulary size to corpus size tends toward zero.
+@c TODO: maybe @samp instead of @verb below; be very careful
+
The performance advantage of @pmg{} arises when different processes
create and access associative arrays. Accessing an element of a
persistent array created by a previous @pmg{} process, as we did
@@ -693,8 +685,8 @@ paging:@footnote{The @command{tee} rigmarole is explained at
@end verbatim
@noindent
Tuning paging parameters can help non-persistent @gwk{} as well as
-@pmg{}. [Disclaimer: OS tuning is an occult art, and your mileage may
-vary.]
+@pmg{}.@*
+[@strong{Disclaimer}: OS tuning is an occult art, and your mileage may vary.]
@c sudo sh -c 'echo 3 > /proc/sys/vm/drop_caches'
@c
@@ -737,9 +729,9 @@ logically enormous sparse file and check its physical size:
@noindent
Whereas @command{ls} reports the logical file size that we expect (one
TiB or 2 raised to the power 40 bytes), @command{du} reveals that the
-file occupies no storage whatsoever. The file system will allocate
-physical storage resources beneath this file as data is written to it;
-reading unwritten regions of the file yields zeros.
+file consumes @emph{zero} storage. The file system will allocate
+physical storage beneath this file as data are written to it; reading
+unwritten regions yields zeros.
The ``pay as you go'' storage cost of sparse files offers both
convenience and control for @pmg{} users. If your file system
@@ -752,26 +744,38 @@ simply set its initial size to whatever bound you wish to
enforce; it
won't eat more disk than that. Copying sparse files with GNU
@command{cp} creates sparse copies by default.
-File-system encryption can preclude sparse files: If the cleartext of
+To maximize storage frugality we sometimes want to ``re-sparsify''
+heap files cluttered with de-allocated memory that @pmg{} no longer
+needs. A stand-alone utility, @command{pma_sam}, is provided for this
+purpose at the @code{pma} web site.
+@c Running @command{du} before & after @command{pma_sam} will quantify
+@c the savings.
+
+@c NOTE: In principle, it would be possible for pm-gawk to
+@c automatically check whether re-sparsification would be
+@c beneficial, and then just do it. However to do this job
+@c well would require knowledge about the file system and
+@c would be quite tricky. It's best to regard issues
+@c surrounding sparse files, including re-sparsification, as
+@c sysadmin matters separate from gawk.
+
+File-system encryption can preclude sparse files: If the plaintext of
a byte offset range within a file is all zero bytes, the corresponding
-ciphertext probably shouldn't be all zeros! Encrypting at the storage
-layer instead of the file system layer may offer acceptable security
-while still permitting file systems to implement sparse files.
-
-Sometimes you might prefer a dense heap file backed by pre-allocated
-storage resources, for example to increase the likelihood that
-@pmg{}'s internal memory allocation will succeed until the persistent
-heap occupies the entire heap file. The @command{fallocate} utility
-will do the trick:
+ciphertext mustn't be all zeros! Encrypting at the storage layer
+instead of the file system may offer acceptable security while still
+permitting sparse files.
+
+Sometimes you want a dense heap file backed by pre-allocated storage,
+e.g., to ensure that @pmg{}'s internal memory allocation will succeed
+until the persistent heap fills the entire file. The
+@command{fallocate} utility does the trick:
@verbatim
$ fallocate -l 1M mibi
$ ls -l mibi
-rw-rw-r--. 1 me me 1048576 Aug 5 23:18 mibi
$ du -h mibi
- 1.0M mibi
+ 1.0M mibi # We get our MiB, both logically & physically.
@end verbatim
-@noindent
-We get the MiB we asked for, both logically and physically.
@c UPDATE: search for username in "ls" examples
@@ -801,7 +805,7 @@ logically distinct concept is the subject of @ref{Data
Integrity}.)
@dfn{Persistent} data outlive the processes that access them, but
don't necessarily last forever. For example, as explained in
-@command{man mq_overview}, message queues are persistent because they
+@samp{man mq_overview}, message queues are persistent because they
exist until the system shuts down. @dfn{Durable} data reside on a
physical medium that retains its contents even without continuously
supplied power. For example, hard disk drives and solid state drives
@@ -840,7 +844,6 @@ Varying the size of random inputs can, for example, find
where
performance ``falls off the cliff'' as @pmg{}'s memory footprint
exceeds the capacity of DRAM and paging begins.
-@c TODO:
@c virtual *machines* / cloud machines can make performance hard to measure
repeatably
@c here we assume good old fashioned OS install directly on "bare metal"
@@ -864,10 +867,8 @@ from cache. Read about @command{sync} and
The C-shell (@command{csh}) script listed below illustrates concepts
and implements tips presented in this chapter. It produced the
results discussed in @ref{Results} in roughly 20 minutes on an aging
-laptop. You can cut and paste the code listing below into a file, or
-download it from @url{http://web.eecs.umich.edu/~tpkelly/pma/}.
-
-@c TODO: post script to Web site when finalized
+laptop. To reproduce my experiments, cut/paste the listing below into
+a file; take care that no lines are duplicated or omitted.
The script measures the performance of four different ways to support
word frequency queries over a text corpus: The na@"{@dotless{i}}ve
@@ -1091,7 +1092,7 @@ memory footprint is very small while serving a query that
accesses a
single array element. The upside of the large heap file is @i{O(1)}
access instead of @i{O(W)}---a classic time-space tradeoff. If
storage is a scarce resource, all three intermediate files can be
-compressed, @code{freqtbl} by a factor of roughly 2.7, @code{rwarray}
+compressed, @code{freqtbl} by a factor of roughly 2.7x, @code{rwarray}
by roughly 5.6x, and @pmg{} by roughly 11x using @command{xz}.
Compression is CPU-intensive and slow, another time-space tradeoff.
@@ -1125,24 +1126,24 @@ Timestamps in backup filenames make it easy to find the
most recent
copy if the heap file is damaged, even if last-mod metadata are
inadvertently altered.
-@c TODO: sync individual files above instead of globally (?)
+@c NOTE: we could sync individual files above instead of globally (???)
@c First carefully check what sync does in both cases
@c using strace, verify that "sync [file]" is correct.
@c Also check whether non-GNU/Linux offers fine-grained
@c sync command. Cygwin? Solaris?
-The @command{cp} command's @command{--reflink} option reduces both the
+The @command{cp} command's @option{--reflink} option reduces both the
storage footprint of the copy and the time required to make it. Just
as sparse files provide ``pay as you go'' storage footprints, reflink
copying offers ``pay as you @emph{change}'' storage
costs.@footnote{The system call that implements reflink copying is
-described in @command{man ioctl_ficlone}.} A reflink copy shares
+described in @samp{man ioctl_ficlone}.} A reflink copy shares
storage with the original file. The file system ensures that
subsequent changes to either file don't affect the other. Reflink
copying is not available on all file systems; XFS, BtrFS, and OCFS2
-currently support it.@footnote{The @command{--reflink} option creates
+currently support it.@footnote{The @option{--reflink} option creates
copies as sparse as the original. If reflink copying is not
-available, @command{--sparse=always} should be used.} Fortunately you
+available, @option{--sparse=always} should be used.} Fortunately you
can install, say, an XFS file system @emph{inside an ordinary file} on
some other file system, such as @code{ext4}.@footnote{See
@url{https://www.usenix.org/system/files/login/articles/login_winter19_08_kelly.pdf}.}
@@ -1200,6 +1201,9 @@ in @ref{Examples}, and also the exercise of making
Kernighan & Pike's
``Markov'' script persistent. Volos provided and tested the advice on
tuning OS parameters in @ref{Virtual Memory and Big Data}. Stan Park
provided insights about virtual memory, file systems, and utilities.
+Antonio Giovanni Colombo translated an earlier edition of this manual
+into Italian in August 2022; see file @file{doc/it/pm-gawk.texi} in
+the @gwk{} distribution tarball.
@c ==================================================================
@c ==================================================================
@@ -1208,37 +1212,114 @@ provided insights about virtual memory, file systems,
and utilities.
@node Installation
@appendix Installation
-@c UPDATE below or remove this section if it's obsolete
-
-@gwk{} 5.2 featuring persistent memory is expected to be released in
-September 2022; look for it at @url{http://ftp.gnu.org/gnu/gawk/}. If
-5.2 is not released yet, the master git branch is available at
-@c
-@url{http://git.savannah.gnu.org/cgit/gawk.git/snapshot/gawk-master.tar.gz}.
-@c
+@c UPDATE this section with every edition of the user manual
+
+Users may obtain @pmg{} via software package management systems or via
+manual installation. Each approach has pros and cons.
+
+@strong{Packages} @w{ } Delegating the chore of installation to
+software package management systems, such as those associated with
+major Linux distributions including Ubuntu and Fedora, is easier---in
+theory. Software packages on some Linux distributions, however, lag
+years behind the latest software releases. Therefore relatively new
+features might be available only in ``upstream'' releases but not
+``downstream'' packages. For example, as of February 2025 the
+package-installed @gwk{} on Ubuntu Linux is pre-5.2 and therefore
+lacks the persistence feature introduced in @gwk{} 5.2, which was
+released way back in September 2022! If the first output line of
+@code{gawk --version} shows 5.2 or later and ``@code{PMA},'' you've
+got a @pmg{} that supports persistence; otherwise see ``Manual
+Install'' below.
+
+@c TODO: Robbins patch with bugzilla below; it says PIE is wrong
+@c (postponing this in Feb '25 because situation is in flux;
+@c it's best to release updated manual when dust settles)
+
+Another problem with software installed by package managers is that
+the software may have been compiled/built incorrectly. For example,
+as of February 2025 the package-installed @gwk{} 5.3.0 on Fedora 41
+was incorrectly created as a position-independent executable (PIE),
+despite the official @gwk{} distribution's build system very
+deliberately and explicitly disabling PIE. The result, noted in
+@ref{Quick Start}, is that @pmg{} crashes the second time it is
+invoked. The details of the Fedora package bug, and remedies that are
+works in progress as of early February 2025, are available at@*
+@w{ } @w{ } @w{ } @w{ } @w{ } @w{ }
@url{https://bugzilla.redhat.com/show_bug.cgi?id=2341653} @*
+The @command{file} utility reveals whether @gwk{} was incorrectly
+built. On Fedora 41, for example:
+@verbatim
+ $ which gawk
+ /usr/bin/gawk
+ $ file /usr/bin/gawk
+ /usr/bin/gawk: ELF 64-bit LSB pie ...
+@end verbatim
+@noindent
+That little word ``@code{pie}'' is likely to blame if your @pmg{}
+crashes on the @emph{second} invocation. By default, PIEs run with
+address-space layout randomization (ASLR), which @gwk{} 5.2 thru the
+current 5.3.1 do not tolerate.
+
+(Note: The persistent memory allocator that enables persistent-memory
+@gwk{}---the @code{pma} library---is perfectly compatible with PIE and
+ASLR. The incompatibility of @gwk{} and ASLR arises from the
+interpreter's internal data structures, which require that function
+pointers be consistent across invocations. ASLR introduces gratuitous
+inconsistencies into these pointers.)
+
+While we're waiting for an elegant and permanent fix for the Fedora 41
+PIE bug, we can use a klugey workaround: Run the @pmg{} interpreter
+via @samp{setarch -R}, which disables ASLR despite the PIE. Here's
+the first example from @ref{Quick Start}, first without the fix, then
+with the fix:
+@verbatim
+ $ truncate -s 4096000 heap.pma
+ $ export GAWK_PERSIST_FILE=heap.pma
+ $ gawk 'BEGIN{myvar = 47}'
+ $ gawk 'BEGIN{myvar += 7; print myvar}'
+ Segmentation fault (core dumped) # thanks to PIE
+
+ $ rm -f heap.pma # discard heap from first try
+ $ truncate -s 4096000 heap.pma # start fresh
+ $ setarch -R gawk 'BEGIN{myvar = 47}'
+ $ setarch -R gawk 'BEGIN{myvar += 7; print myvar}'
+ 54 # success
+@end verbatim
+@noindent
+You can define a shell alias to expand the familiar and ergonomic
+@gwk{} into the cumbersome and verbose @samp{setarch -R gawk}.
+
+Alternatively, it might be possible to disable ASLR system-wide by
+using @code{sysctl} to twiddle variables such as
+@code{kernel.randomize_va_space}. I have not investigated such
+measures, which should be used with caution. Fully understand the
+implications of such a blunt system-wide change before making it.
+
+It's reasonable to expect that, in the normal course of events,
+correctly built @pmg{} will eventually find its way into the default
+package-installed @gwk{} on major GNU/Linux distros. Meanwhile,
+manual installation is a fairly easy way to get a working @pmg{}.
+
+@strong{Manual Install} @w{ } Manually compiling @gwk{} from source
+code in the latest upstream release gives you the most recent stable
+version of @gwk{}, built in accordance with with the maintainer's
+recipe, with no meddling by intermediaries. Therefore a manual
+install won't suffer from bugs such as the PIE bug discussed above.
+Download the latest release here:@*
+@w{ } @w{ } @w{ } @w{ } @w{ } @w{ } @url{https://ftp.gnu.org/gnu/gawk/}@*
+@w{ } @w{ } @w{ } @w{ } @w{ } @w{ }
@url{https://ftp.gnu.org/gnu/gawk/gawk-5.3.1.tar.xz}
+
+Finally, for the ultimate in bleeding-edge upstream freshness,
+adventurous do-it-yourselfers can grab the @command{git} master branch
+from@*
+@url{http://git.savannah.gnu.org/cgit/gawk.git/snapshot/gawk-master.tar.gz}@*
Unpack the tarball, run @command{./bootstrap.sh},
@command{./configure}, @command{make}, and @command{make check}, then
-try some of the examples presented earlier. In the normal course of
-events, 5.2 and later @gwk{} releases featuring @pmg{} will appear in
-the software package management systems of major GNU/Linux distros.
-Eventually @pmg{} will be available in the default @gwk{} on such
-systems.
-
-@c ADR comments on above, "run ./bootstrap.sh, ./configure ..."
-@c TK replies: I haven't been doing this. Neither the README nor the
-@c INSTALL in the gawk tarball mention bootstrap.sh. If it's
-@c important, shouldn't they? The top of bootstrap.sh says its
-@c purpose is "to avoid out-of-date issues in Git sandboxes."
-@c When a neurodivergent source code control system requires us to
-@c write shell scripts to work around the problems that it creates
-@c gratuitously, the universe is trying to tell us something about
-@c it.
-
-@c official gawk:
-@c http://ftp.gnu.org/gnu/gawk/
[where to look for 5.2 after release]
-@c https://www.skeeve.com/gawk/gawk-5.1.62.tar.gz
[doesn't support persistent functions]
-@c http://git.savannah.gnu.org/cgit/gawk.git/snapshot/gawk-master.tar.gz
[if 5.2 isn't released yet]
-@c http://git.savannah.gnu.org/cgit/gawk.git
[ongoing development]
+try some of the examples presented earlier.
+
+As of February 2025, @pmg{} is supported on most, but not all, major
+Unix-like platforms. The @gwk{} build system decides whether to
+include support for persistence; see the @gwk{} documentation and file
+@file{./m4/pma.m4} in the @gwk{} distribution.
@c ==================================================================
@node Debugging
@@ -1259,16 +1340,18 @@ AWK variables such as @code{NR} are always reset to
zero every time
the interpreter runs. See the discussion of initialization
surrounding the min/max/mean script in @ref{Examples}.
-If you suspect a persistence-related bug in @pmg{}, you can set
-an environment variable that will cause its persistent heap module,
-@code{pma}, to emit more verbose error messages; for details see the
-main @gwk{} documentation.
+If @pmg{} crashes on the @emph{second} invocation that uses a
+particular heap file, see the discussion of PIE in @ref{Installation}.
+If you suspect some other kind of persistence-related bug in @pmg{},
+you can set an environment variable that will cause its persistent
+heap module, @code{pma}, to emit more verbose error messages; for
+details see the main @gwk{} documentation.
@c or the @code{pma} documentation at
@c @url{http://web.eecs.umich.edu/~tpkelly/pma/}.
Programmers: You can re-compile @gwk{} with assertions enabled, which
will trigger extensive integrity checks within @code{pma}. Ensure
-that @file{pma.c} is compiled @emph{without} the @code{-DNDEBUG} flag
+that @file{pma.c} is compiled @emph{without} the @option{-DNDEBUG} flag
when @command{make} builds @gwk{}. Run the resulting executable on small
inputs, because the integrity checks can be very slow. If assertions
fail, that likely indicates bugs somewhere in @pmg{}. Report such
@@ -1277,7 +1360,6 @@ main @gwk{} documentation. Specify what version of
@gwk{} you're
using, and try to provide a small and simple script that reliably
reproduces the bug.
-@page
@c ==================================================================
@node History
@appendix History
@@ -1288,7 +1370,7 @@ allocator, @code{pma}, whose design is described in
to trace the evolutionary paths that led to @code{pma} and @pmg{}.
I wrote many AWK scripts during my dissertation research on Web
-caching twenty years ago, most of which processed log files from Web
+caching 25 years ago, most of which processed log files from Web
servers and Web caches. Persistent @gwk{} would have made these
scripts smaller, faster, and easier to write, but at the time I was
unable even to imagine that @pmg{} is possible. So I wrote a lot of
@@ -1323,7 +1405,7 @@ the Linux kernel, calling the result ``failure-atomic
@code{msync()}''
memory-mapped file always reflects the most recent successful
@code{msync()} call, even in the presence of failures such as power
outages and OS or application crashes. The original Linux kernel FAMS
-prototype is described in a paper by Park et al. in EuroSys 2013. My
+prototype is described in a paper by Park et al.@: in EuroSys 2013. My
colleagues and I subsequently implemented FAMS in several different
ways including in file systems (FAST 2015) and user-space libraries.
My most recent FAMS implementation, which leverages the reflink
@@ -1333,22 +1415,21 @@ ubiquitous GNU @command{dbm} (@command{gdbm}) database
(@url{https://queue.acm.org/detail.cfm?id=3487353}).
In recent years my attention has returned to the advantages of
-persistent memory programming, lately a hot topic thanks to the
-commercial availability of byte-addressable non-volatile memory
-hardware (which, confusingly, is nowadays marketed as ``persistent
-memory''). The software abstraction of persistent memory and the
-corresponding programming style, however, are perfectly compatible
-with @emph{conventional} computers---machines with neither
-non-volatile memory nor any other special hardware or software. I
-wrote a few papers making this point, for example
-@url{https://queue.acm.org/detail.cfm?id=3358957}.
+persistent memory programming, which was a hot topic circa COVID
+thanks to the fleeting commercial availability of byte-addressable
+non-volatile memory hardware (Intel Optane, confusingly marketed as
+``persistent memory''). The software abstraction of persistent memory
+and the corresponding programming style, however, are perfectly
+compatible with @emph{conventional} computers---machines with neither
+non-volatile memory nor any other special hardware or software.
+Several papers make this point, e.g.,
+@url{https://queue.acm.org/detail.cfm?id=3358957}
In early 2022 I wrote a new stand-alone persistent memory allocator,
@code{pma}, to make persistent memory programming easy on conventional
hardware. The @code{pma} interface is compatible with @code{malloc()}
and, unlike Ken's allocator, @code{pma} is not coupled to a particular
-crash-tolerance mechanism. Using @code{pma} is easy and, at least to
-some, enjoyable.
+crash-tolerance mechanism. Using @code{pma} is easy and fun.
Ken had been integrated into prototype forks of both the V8 JavaScript
interpreter and a Scheme interpreter, so it was natural to consider
@@ -1378,7 +1459,7 @@ available at
@c open source offers more impact than research
@c work with colleagues who Think Different from one another
-@sp 2
+@c @sp 2
I enjoy several aspects of @pmg{}. It's unobtrusive; as you gain
familiarity and experience, it fades into the background of your
@@ -1386,7 +1467,7 @@ scripting. It's simple in both concept and
implementation, and more
importantly it simplifies your scripts; much of its value is measured
not in the code it enables you to write but rather in the code it lets
you discard. It's all that I needed for my dissertation research
-twenty years ago, and more. Anecdotally, it appears to inspire
+25 years ago, and more. Anecdotally, it appears to inspire
creativity in early adopters, who have devised uses that @pmg{}'s
designers never anticipated. I'm curious to see what new purposes
you find for it.
-----------------------------------------------------------------------
Summary of changes:
doc/ChangeLog | 10 ++
doc/gawk.info | 183 +++++++++++++++--------------
doc/gawk.texi | 8 +-
doc/pm-gawk.info | 298 ++++++++++++++++++++++++++++++-----------------
doc/pm-gawk.texi | 349 ++++++++++++++++++++++++++++++++++---------------------
5 files changed, 521 insertions(+), 327 deletions(-)
hooks/post-receive
--
gawk
| [Prev in Thread] |
Current Thread |
[Next in Thread] |
- [SCM] gawk branch, master, updated. gawk-4.1.0-5782-gb7300af5,
Arnold Robbins <=