Re: [PATCH] Implement open-process and related functions on MinGW

[Eli Zaretskii]

> From: Mark H Weaver <address@hidden>
> Cc: address@hidden,  address@hidden
> Date: Sun, 23 Feb 2014 00:50:16 -0500
> 
> Eli Zaretskii <address@hidden> writes:
> 
> >> Definitions for these on MinGW are available in Gnulib's 'sys_wait'
> >> module.  I'll import it soon.
> >
> > Please don't: that gnulib module is dead wrong for Windows.  If we use
> > it, we will be unable to report even the simplest error exits, let
> > alone programs that crashed due to a fatal signal.
> 
> Hmm.  This is rather uncomfortable.  Did you discuss these problems with
> the Gnulib developers?  If so, can you provide a link to the relevant
> threads?

I'm sorry, but I seem to be unable to efficiently discuss with gnulib
developers anything that is related to MinGW and Windows in general.
Any such discussion at best drags on for months, and at worst simply
dies because there's no response.  See a few examples below.  And no,
I didn't discuss this specific issue with gnulib developers.

> Your comment above "We usurp codes above 0xC0000200 for SIGxxx signals"
> makes me wonder: How widely used is this convention?  What other
> software packages use it?

I don't think this matters.  This convention is used between Guile and
itself: in scm_kill we tell the process being killed to terminate with
status of 0xC0000200 + SIGNAL, and then we extract the signal number
in status:term-sig.  So this is just something private to Guile,
that's all; its only purpose is to report the signal with which we
killed the process.  Other applications don't need to know about this.

> My reasons for this preference are (1) to keep the Guile code cleaner,
> and (2) if some deficiency is found in the wrapper, or if Windows
> introduces an improved API some day, the changes have to be made in only
> one place: Gnulib.

As long as this is practical, I won't argue with you.  However, my
experience with gnulib is significantly different (see below), and
therefore my personal preferences are also different.

> > But in general, due to the known problems with getting
> > Windows-specific patches to gnulib approved and admitted, I'd advise
> > to stay away of gnulib as much as possible, when Windows is concerned.
> 
> Can you provide some links to mailing list threads where you've tried to
> get things fixed in Gnulib and encountered resistance?

Certainly.  My first example is with canonicalize_filename, which
started here:

  http://lists.gnu.org/archive/html/bug-gnulib/2012-01/msg00253.html

and then promptly died.  My pings here

  http://lists.gnu.org/archive/html/bug-gnulib/2012-06/msg00222.html
  http://lists.gnu.org/archive/html/bug-gnulib/2012-07/msg00004.html
  http://lists.gnu.org/archive/html/bug-gnulib/2012-07/msg00239.html

were left without any response whatsoever.  Finally, you intervened
here:

  http://lists.gnu.org/archive/html/bug-gnulib/2012-11/msg00021.html

and, lo and behold, 3 weeks later the changes were finally committed,
more than 10 months after the initial submittal.

This is a successful story, mind you.  Here's a couple of unsuccessful
ones:

  http://lists.gnu.org/archive/html/bug-gnulib/2013-06/msg00042.html
  http://lists.gnu.org/archive/html/bug-gnulib/2014-01/msg00072.html

Who knows, perhaps a year from now these will also become "successes"?
One can hope, can't one?

> > E.g., what if tomorrow we would need to make the waitpid emulation
> > more sophisticated?
> 
> I suppose I would prefer to get the problem fixed in Gnulib.  If there
> is truly a problem with the way Gnulib is being maintained w.r.t. to
> Windows, perhaps I will make an effort to get that larger problem fixed.

See above.

> For example, if it is true that we can avoid all the nasty problems with
> filename encoding by using the native Windows APIs that use UTF-16 for
> filenames, then I'd be in favor of doing that.

What nasty problems do you have in mind?  I implemented something like
this in Emacs not long ago, so perhaps I can help here.

> > In general, where the Windows equivalent is to do nothing, my advice
> > would be to have a no-op implementation, rather than an unbound
> > function.  The former approach makes it much easier to write portable
> > Guile scripts, instead of spreading boundness checks all over the
> > place.
> 
> I think this is a very bad idea.  First of all, it's not very common for
> Guile programs to query or set the uid/gid, so I don't see much benefit.

Unless you remove them from Guile, I don't think this argument stands:
as long as the functions exist, some Guile program will certainly use
them.  For example, those servers where security is important, as you
mention.

> On the other hand, if a program _does_ try to do one of those things, it
> might be important that the job be done right.  For example, a program
> might be trying to reduce its privileges.  Doing nothing while silently
> pretending that the operation succeeded is a good way to cause security
> flaws to go unnoticed, or to make a programmer waste hours of time
> trying to debug a problem that he should have been notified about
> immediately with an error message.

We are talking about operations that have no equivalent meaning on
Windows.  they only have meaning in Posix worldview.  Why does it make
sense to fail an operation that is meaningless?  What useful purpose
could this possibly serve, except having a subtly broken Guile module?

> > FWIW, Emacs takes the former approach with very good results.
> 
> Emacs is primarily an interactive editor

Oh, Emacs is much more than that, believe me.

> Guile is a general-purpose programming language, and we must
> consider the needs of a larger set of users, for example the authors
> of server software where security is important.

Server software cannot be written portably anyway.  I can believe you
that Guile has enough infrastructure to support writing a server on
Posix platforms, but it certainly does _not_ have the infrastructure
for doing that on Windows; using setuid etc., or setting privileges,
will be the least of your worries, if you try doing that.  So this is
a red herring, I think.  By contrast, calls to setuid are quite common
in modern programs that needs to run on Posix platforms.

> >> Can you use the GNU coding conventions for placement of brackets?
> >
> > Sorry, I don't follow: which part is not according to the GNU coding
> > conventions?
> 
> Nevermind, my mistake.  Your use of tabs in the indentation combined
> with the prefix characters in the email caused some (but not all) lines
> to move to the right.  I've long been using (setq indent-tabs-mode nil)
> globally, for that reason.

Using tabs like I did is the Emacs default.  Is there a requirement in
Guile to use only spaces?  If so, I probably missed it.

> >> Thanks for working on this, but in a multithreaded program, it's no good
> >> to change the file descriptors in the main program temporarily before
> >> spawning, and then restore them afterwards.  We'll have to find another
> >> way of doing this.
> >
> > Threads don't work in the MinGW build anyway, and no one was able to
> > help me understand why (see the discussions last August).  As long as
> > this limitation exists, this is a non-issue, and certainly better than
> > not having this functionality available at all.
> 
> As I wrote elsewhere, madsy on #guile reported having successfully built
> a recent Guile snapshot with both threads and posix enabled.
> 
> I know that you asked for more specifics about this, but I don't know
> the answers.  I asked madsy to reply to you directly when he has some
> free time.  However, I remember that he cross-built from Ubuntu

Cross-building on GNU/Linux is probably the cause of success: the
problematic steps run on a Posix platform using Posix APIs, I presume?
Otherwise, the issues are subtle and only show when running certain
Guile programs that use threading infrastructure, as discussed and
described back in August.

> and iirc he built libgc (and several other dependencies) from source
> code.

I've built libgc myself as well (and it passed all the tests), so this
cannot be the reason.  I also built libgmp.
     
> > The only other way I know of is to bypass Posix-like functions like
> > 'open', 'close', and 'dup', and go to the level of Win32 API, where a
> > function that creates a child subprocess can accept handles for the
> > child's standard I/O.
> 
> This sounds to me like the right approach in this case.

OK.

> > If you will be comfortable with a lot more Widows specific stuff, I
> > can work on that.
> 
> Please do!

OK, but please don't block this patch while you wait for me to do
that.  It is certainly much better to have this functionality with
restrictions than not at all.  At the very least, we could condition
redirection of standard handles on !SCM_USE_PTHREAD_THREADS, and
otherwise support only child processes that don't require redirection.
This is already much better than what we have today, even if Guile
with threads is indeed workable on Windows.