Re: Extension packaging

[Eli Zaretskii]

> Date: Tue, 10 May 2022 19:54:29 +0300
> From: Eli Zaretskii <eliz@gnu.org>
> Cc: mcollado2011@gmail.com, pjfarley3@earthlink.net, bug-gawk@gnu.org
> 
> > My initial instructions were sloppy. They should have said:
> > 
> > wget 
> > https://sourceforge.net/projects/gawkextlib/files/gawkextlib-1.0.4.tar.gz
> > tar xf gawkextlib-1.0.4.tar.gz
> > cd gawkextlib-1.0.4.tar.gz
> > ./configure && make && make check && make install && echo Success.
> > cd ..
> > wget 
> > https://sourceforge.net/projects/gawkextlib/files/gawk-mpfr-1.1.0.tar.gz
> > tar xf gawk-mpfr-1.1.0.tar.gz
> > cd gawk-mpfr-1.1.0
> > ./configure && make && make check && make install && echo Success.
> 
> OK, will do.  However, ...
> 
> > But I am curious as to whether you can get this to work on MinGW if you're
> > willing to waste a few more minutes on this.
> 
> I will try and report.

With these instructions, it works, more or less.  Findings:

  . gawkextlib builds, and its "make install-strip" does TRT.
  . there were some warnings during the build which I ignored; see
    below.
  . gawk-mpfr also builds and produces mpfr.dll, as expected.
  . "make install" in gawk-mpfr installs the shared library not where
    it should be, which is expected, per our previous discussion.

Then I tried "make check" in gawk-mpfr, which failed:

  make[1]: Entering directory `/d/gnu/gawkextlib/gawk-mpfr-1.1.0/test'
  ======== Starting MPFR extension tests ========
  xmpfrmat
  ./xmpfrmat.ok _xmpfrmat differ: char 1, line 1
  Makefile:533: recipe for target `xmpfrmat' failed
  make[1]: [xmpfrmat] Error 1 (ignored)
  xmpfrall
  ./xmpfrall.ok _xmpfrall differ: char 1, line 1
  Makefile:538: recipe for target `xmpfrall' failed
  make[1]: [xmpfrall] Error 1 (ignored)
  ======== Done with MPFR extension tests ========
  make[2]: Entering directory `/d/gnu/gawkextlib/gawk-mpfr-1.1.0/test'
  2 TESTS FAILED

The "char 1, line 1" thing rang a bell, so I looked into
test/Makefile, and sure enough: it uses 'cmp' for comparison to
expected results, which doesn't work on MS-Windows because of the
CR-LF vs newline differences in EOL format.  Next attempt:

  $ make check CMP='diff -u'
  ======== Starting MPFR extension tests ========
  xmpfrmat
  --- ./xmpfrmat.ok       2014-11-01 17:29:21 +0200
  +++ _xmpfrmat   2022-05-11 15:20:31 +0300
  @@ -1 +1,37 @@
  -1.0000000000000000 2.5700000000000000E2
  +Usage: gawk.exe [POSIX or GNU style options] -f progfile [--] file ...
  +Usage: gawk.exe [POSIX or GNU style options] [--] 'program' file ...
  +POSIX options:         GNU long options:
  +       -f progfile             --file=progfile
  +       -F fs                   --field-separator=fs
  +       -v var=val              --assign=var=val
  +       -m[fr] val
  +       -O                      --optimize
  +       -W compat               --compat
  +       -W copyleft             --copyleft
  +       -W copyright            --copyright
  +       -W dump-variables[=file]        --dump-variables[=file]
  +       -W exec=file            --exec=file
  +       -W gen-po               --gen-po
  +       -W help                 --help
  +       -W lint[=fatal]         --lint[=fatal]
  +       -W lint-old             --lint-old
  +       -W non-decimal-data     --non-decimal-data
  +       -W profile[=file]       --profile[=file]
  +       -W posix                --posix
  +       -W re-interval          --re-interval
  +       -W source=program-text  --source=program-text
  +       -W traditional          --traditional
  +       -W usage                --usage
  +       -W use-lc-numeric       --use-lc-numeric
  +       -W version              --version
  +
  +To report bugs, see node `Bugs' in `gawk.info', which is
  +section `Reporting Problems and Bugs' in the printed version.
  +
  +gawk is a pattern scanning and processing language.
  +By default it reads standard input and writes standard output.
  +
  +Examples:
  +       gawk '{ sum += $1 }; END { print sum }' file
  +       gawk -F: '{ print $1 }' /etc/passwd
  +EXIT CODE: 1
  Makefile:533: recipe for target `xmpfrmat' failed
  make[1]: [xmpfrmat] Error 1 (ignored)

This is a sure sign that a wrong Gawk is being invoked (test/Makefile
says "GAWKPROG = gawk.exe" even though I configured --with-gawk= to
point to the correct binary).

Next attempt:

  $ make check CMP='diff -u' GAWKPROG=/d/usr/bin/gawk.exe
  ======== Starting MPFR extension tests ========
  xmpfrmat
  --- ./xmpfrmat.ok       2014-11-01 17:29:21 +0200
  +++ _xmpfrmat   2022-05-11 15:21:47 +0300
  @@ -1 +1,2 @@
  -1.0000000000000000 2.5700000000000000E2
  +gawk: fatal: cannot open shared library `mpfr' for reading: No such file or 
directory
  +EXIT CODE: 2
  Makefile:533: recipe for target `xmpfrmat' failed

This happens because of this gork in test/Makefile:

  AWK = LC_ALL=$${GAWKLOCALE:-C} LANG=$${GAWKLOCALE:-C} 
AWKLIBPATH=../.libs:`$(GAWKPROG) 'BEGIN {print ENVIRON["AWKLIBPATH"]}'` [...]

The MinGW Gawk needs AWKLIBPATH with semi-colons, not colons.  The
MSYS Bash I use converts colons to semi-colons before it invokes Gawk,
but it gets confused because 'BEGIN {print ENVIRON["AWKLIBPATH"]}'
produces a Windows-style d:/foo/bar file name with a colon after the
drive letter, so that colon also gets converted to a semi-colon.
Solution: a Sed script:

  AWK = LC_ALL=$${GAWKLOCALE:-C} LANG=$${GAWKLOCALE:-C} 
AWKLIBPATH=../.libs:`$(GAWKPROG) 'BEGIN {print ENVIRON["AWKLIBPATH"]}' | sed 
"s,d:/,/d/,"`

And now all the tests pass:

  $ make check CMP='diff -u' GAWKPROG=/d/usr/bin/gawk.exe
  ======== Starting MPFR extension tests ========
  xmpfrmat
  xmpfrall
  ======== Done with MPFR extension tests ========
  make[1]: Entering directory `/d/gnu/gawkextlib/gawk-mpfr-1.1.0/test'
  ALL TESTS PASSED
  make[1]: Leaving directory `/d/gnu/gawkextlib/gawk-mpfr-1.1.0/test'

Conclusions:

  . building gawk-mpfr with MinGW is relatively easy
  . it would be nice to have in README something to the effect that
    gawkextlib should be built and installed first
  . "make install" in gawk-mpfr Needs Work(TM)
  . the test suite needs some quirks to run on MS-Windows

Here are the warnings I saw while compiling gawkextlib (with GCC 9.2.0):

  libtool: compile:  gcc -DHAVE_CONFIG_H -I. -Id:/usr/bin/gawk -g -O2 -Wall 
-Wextra -Wpointer-arith -Wmissing-prototypes -Wcast-qual -Wwrite-strings 
-Wshadow -MT strhash.lo -MD -MP -MF .deps/strhash.Tpo -c strhash.c  
-DDLL_EXPORT -DPIC -o .libs/strhash.o
  strhash.c: In function 'my_hash':
  strhash.c:104:5: warning: this statement may fall through 
[-Wimplicit-fallthrough=]
    104 |   h = *s++ + htmp + (htmp << 10) - h
        |   ~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  strhash.c:115:5: note: in expansion of macro 'HASHC'
    115 |     HASHC;
        |     ^~~~~
  strhash.c:116:3: note: here
    116 |   case 7:  HASHC;
        |   ^~~~
  strhash.c:104:5: warning: this statement may fall through 
[-Wimplicit-fallthrough=]
    104 |   h = *s++ + htmp + (htmp << 10) - h
        |   ~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  strhash.c:116:12: note: in expansion of macro 'HASHC'
    116 |   case 7:  HASHC;
        |            ^~~~~
  strhash.c:117:3: note: here
    117 |   case 6:  HASHC;
        |   ^~~~
  strhash.c:104:5: warning: this statement may fall through 
[-Wimplicit-fallthrough=]
    104 |   h = *s++ + htmp + (htmp << 10) - h
        |   ~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  strhash.c:117:12: note: in expansion of macro 'HASHC'
    117 |   case 6:  HASHC;
        |            ^~~~~
  strhash.c:118:3: note: here
    118 |   case 5:  HASHC;
        |   ^~~~
  strhash.c:104:5: warning: this statement may fall through 
[-Wimplicit-fallthrough=]
    104 |   h = *s++ + htmp + (htmp << 10) - h
        |   ~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  strhash.c:118:12: note: in expansion of macro 'HASHC'
    118 |   case 5:  HASHC;
        |            ^~~~~
  strhash.c:119:3: note: here
    119 |   case 4:  HASHC;
        |   ^~~~
  strhash.c:104:5: warning: this statement may fall through 
[-Wimplicit-fallthrough=]
    104 |   h = *s++ + htmp + (htmp << 10) - h
        |   ~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  strhash.c:119:12: note: in expansion of macro 'HASHC'
    119 |   case 4:  HASHC;
        |            ^~~~~
  strhash.c:120:3: note: here
    120 |   case 3:  HASHC;
        |   ^~~~
  strhash.c:104:5: warning: this statement may fall through 
[-Wimplicit-fallthrough=]
    104 |   h = *s++ + htmp + (htmp << 10) - h
        |   ~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  strhash.c:120:12: note: in expansion of macro 'HASHC'
    120 |   case 3:  HASHC;
        |            ^~~~~
  strhash.c:121:3: note: here
    121 |   case 2:  HASHC;
        |   ^~~~
  strhash.c:104:5: warning: this statement may fall through 
[-Wimplicit-fallthrough=]
    104 |   h = *s++ + htmp + (htmp << 10) - h
        |   ~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  strhash.c:121:12: note: in expansion of macro 'HASHC'
    121 |   case 2:  HASHC;
        |            ^~~~~
  strhash.c:122:3: note: here
    122 |   case 1:  HASHC;
        |   ^~~~