Re: GNU make 4.3.90 release candidate available

[Eli Zaretskii]

> From: Paul Smith <psmith@gnu.org>
> Cc: bug-make@gnu.org
> Date: Mon, 26 Sep 2022 14:23:40 -0400
> 
> On Mon, 2022-09-26 at 20:23 +0300, Eli Zaretskii wrote:
> > The following tests should be disabled on MS-Windows, because they
> > cannot possibly work:
> 
> I have access to a VM running Windows 10 with a relatively recent
> Visual Studio install, with a recent Perl, and with the Git for Windows
> binaries (from MinGW I assume) on my PATH.  No other cygwin or MinGW.

Some binaries from Git for Windows are MinGW binaries, but some are
MSYS2 binaries.  All those for which Git Bash says they are in /bin
(as opposed to /mingw64/bin) are MSYS2 binaries, they depend on MSYS
DLL and don't use the Windows C runtime.  MSYS2 is a fork of Cygwin,
with a few subtle features that allow it to invoke native Windows
programs without confusing them with Unix-style file names and PATH.

> In this environment I can do this (in a Windows cmd.exe shell):
> 
>   tar xzf make-4.3.90.tar.gz
>   cd make-4.3.90
>   .\build_w32.bat
>   .\WinRel\gnumake.exe check
> 
> This builds with Visual Studio and I get zero test failures.
> 
> I assume a big part of this is that make is choosing bash as its shell,
> from the Git for Windows install, not cmd.exe.  Plus other POSIX tools
> are available.

Of course, it's because you set up Bash as the shell Make uses!  Which
means many tests run in an environment that will rarely happen in Real
Life, when the native port of Make is used with cmd.exe as the shell,
not with MSYS2 Bash.

The tests do need a few Unix tools (like 'sleep', 'cat', 'touch', and
maybe others), but those are available as native MinGW ports; whereas
Bash isn't.  Moreover, when a Makefile uses the MSYS2 Bash, it will
generally invoke MSYS2 binaries of other tools, like 'cat' and 'touch'
(unless you are very careful to set up your PATH for Git Bash).  So
you end up using "foreign" tools to run the tests, and I find that
less than optimal.  (It is, of course, much less time-consuming ;-)

> >  . 1 test in misc/bs-nl that uses single quotes around newlines
> >  . 1 test in misc/general3 that uses single quotes around newlines
> >  . 1 test in misc/general4 that unsets PATH (this cannot work on
> >    Windows because Make will be unable to find the DLLs against which
> >    it was linked, so it fails to start without valid PATH)
> >  . targets/ONESHELL in its entirety (this feature doesn't work on
> >    Windows)
> 
> I assume most/all of the above really mean, these tests won't work on
> Windows when make uses a Windows shell such as cmd.exe.

The last two should fail with MSYS Bash as well, I think.

> Maybe we need to add a new check to the test framework that sets a
> variable depending on what kind of shell make is using, then this
> variable should be used when deciding which tests to run rather than
> using "are we on Windows or not".

Feel free, but what's the point of artificially making the tests
"succeed" by forcing them to run in the MSYS environment?  What does
it gain us for testing the native MS-Windows port of Make?  MSYS has
its own port of GNU Make, one that uses largely the Posix code paths
and bypasses the Windows-specific parts (for example, they use the
job-server for parallelism).

> >  . targets/POSIX in its entirety
> 
> The .POSIX: special target tells make that it should modify its
> behavior to conform to the POSIX specification of make, in places where
> it might differ by default.  This happens on Windows the same as on
> non-Windows.

Yes, but look what is being tested there.  I could, of course, tweak
those tests to succeed on native Windows, but what's the point?  E.g.,
this:

  # Test the default value of various POSIX-specific variables
  my %POSIX = (AR => 'ar', ARFLAGS => '-rv',
               YACC => 'yacc', YFLAGS => '',
               LEX => 'lex', LFLAGS => '',
               LDFLAGS => '',
               CC => 'c99', CFLAGS => '-O1',
               FC => 'fort77', FFLAGS => '-O1',
               SCCSFLAGS => '', SCCSGETFLAGS => '-s');

Does it really make sense to jump through hoops in order to test this
on Windows?