Re: [PATCH] Implement open-process and related functions on MinGW

[Mark H Weaver]

Eli Zaretskii <address@hidden> writes:

>> From: Mark H Weaver <address@hidden>
>> Cc: address@hidden,  address@hidden
>> Date: Sun, 23 Feb 2014 00:50:16 -0500
>> 
>> Eli Zaretskii <address@hidden> writes:
>> 
>> >> Definitions for these on MinGW are available in Gnulib's 'sys_wait'
>> >> module.  I'll import it soon.
>> >
>> > Please don't: that gnulib module is dead wrong for Windows.  If we use
>> > it, we will be unable to report even the simplest error exits, let
>> > alone programs that crashed due to a fatal signal.
>> 
>> Hmm.  This is rather uncomfortable.  Did you discuss these problems with
>> the Gnulib developers?  If so, can you provide a link to the relevant
>> threads?
>
> I'm sorry, but I seem to be unable to efficiently discuss with gnulib
> developers anything that is related to MinGW and Windows in general.
> Any such discussion at best drags on for months, and at worst simply
> dies because there's no response.  See a few examples below.  And no,
> I didn't discuss this specific issue with gnulib developers.
>
>> Your comment above "We usurp codes above 0xC0000200 for SIGxxx signals"
>> makes me wonder: How widely used is this convention?  What other
>> software packages use it?
>
> I don't think this matters.  This convention is used between Guile and
> itself: in scm_kill we tell the process being killed to terminate with
> status of 0xC0000200 + SIGNAL, and then we extract the signal number
> in status:term-sig.  So this is just something private to Guile, [...]

I don't like this.  I think our 'status:*' procedures should follow a
common convention, or simply punt on it if there's no convention.

The Gnulib sys_wait module says in its comments:

  "When an unhandled fatal signal terminates a process,
   the exit code is 3."

If this is wrong, can you help me understand what they might have been
thinking when they wrote it?

The Gnulib comments also say:

  "The signal that terminated a process is not known posthum."

and on that basis, they make WTERMSIG(x) simply return SIGTERM.  To my
mind, this seems better than making up our own convention that nobody
else follows, and assuming that the subprocess is another guile process.

If you think that the Gnulib sys_wait module is "dead wrong for
Windows", then please post a message to the Gnulib mailing list and make
your case.  If there's truly a problem, then we need to get it fixed in
Gnulib.  I'd also like to see how they respond to your claims.

>> For example, if it is true that we can avoid all the nasty problems with
>> filename encoding by using the native Windows APIs that use UTF-16 for
>> filenames, then I'd be in favor of doing that.
>
> What nasty problems do you have in mind?  I implemented something like
> this in Emacs not long ago, so perhaps I can help here.

The nasty problem is that POSIX uses sequences of bytes for filenames,
although conceptually filenames are character strings, and in fact
virtually all user interfaces treat filenames as character strings.

Guile uses character strings for filenames (the only sane thing to do),
and it would be good to build Guile on system APIs that also use
character strings for filenames, instead of having to guess how to
encode the characters into bytes.

We don't have a fully satisfactory solution to this problem on POSIX,
but I guess we do on Windows, if we use the native Windows APIs.

BTW, the same problems exist for command-line arguments, environment
variables, the hostname, etc.  All of these are sequences of bytes in
POSIX, but conceptually they should be character strings.

If you'd like to work on a patch to have Guile use the native Windows
APIs (that use UTF-16) for these things, I think that would be very
useful and worthy of inclusion.

>> > In general, where the Windows equivalent is to do nothing, my advice
>> > would be to have a no-op implementation, rather than an unbound
>> > function.  The former approach makes it much easier to write portable
>> > Guile scripts, instead of spreading boundness checks all over the
>> > place.
>> 
>> I think this is a very bad idea.  First of all, it's not very common for
>> Guile programs to query or set the uid/gid, so I don't see much benefit.
[...]
>> On the other hand, if a program _does_ try to do one of those things, it
>> might be important that the job be done right.  For example, a program
>> might be trying to reduce its privileges.  Doing nothing while silently
>> pretending that the operation succeeded is a good way to cause security
>> flaws to go unnoticed, or to make a programmer waste hours of time
>> trying to debug a problem that he should have been notified about
>> immediately with an error message.
>
> We are talking about operations that have no equivalent meaning on
> Windows.  they only have meaning in Posix worldview.  Why does it make
> sense to fail an operation that is meaningless?  What useful purpose
> could this possibly serve, except having a subtly broken Guile module?

As I said before, useful purposes include (1) to alert the user to a
potential security flaw, and (2) to save someone from a possibly long
debugging session when they should have gotten an error message.

In general, I think it's a serious mistake to assume that what the user
asked to do was unimportant, and can therefore be silently ignored.

Does Windows really not have a concept of different security contexts,
with different privileges?  I find that hard to believe, but if it's
true, then I could perhaps be convinced that we should make up a uid/gid
when the user asks for one.

However, I will strongly oppose turning setuid or setgid into silent
no-ops.

>> Emacs is primarily an interactive editor
>
> Oh, Emacs is much more than that, believe me.

Yes, I know that.  You're talking to someone who does almost everything
inside of Emacs.  I run my interactive shells in shell-mode, use gnus
for mail/news, info and woman for reading docs, docview for viewing
PDFs, emacs-w3m for web browsing, erc+bitlbee for irc+jabber, magit for
git, sound editing with my own custom elisp mode, etc.  It's actually
quite rare for any other window to be open in my X session.

> Using tabs like I did is the Emacs default.  Is there a requirement in
> Guile to use only spaces?  If so, I probably missed it.

No, it's not currently a requirement for C code, but IMO the Emacs
default should be changed.

>> >> Thanks for working on this, but in a multithreaded program, it's no good
>> >> to change the file descriptors in the main program temporarily before
>> >> spawning, and then restore them afterwards.  We'll have to find another
>> >> way of doing this.
>> >
>> > Threads don't work in the MinGW build anyway, and no one was able to
>> > help me understand why (see the discussions last August).  As long as
>> > this limitation exists, this is a non-issue, and certainly better than
>> > not having this functionality available at all.
>> 
>> As I wrote elsewhere, madsy on #guile reported having successfully built
>> a recent Guile snapshot with both threads and posix enabled.
>> 
>> I know that you asked for more specifics about this, but I don't know
>> the answers.  I asked madsy to reply to you directly when he has some
>> free time.  However, I remember that he cross-built from Ubuntu

I've since learned more details about what he did.  He built
pthreads-w32 2.9.1 <https://www.sourceware.org/pthreads-win32/>
from source code.  He also built libgc-7.2e.  He said that the
MinGW cross-compiler identified itself as follows:

"i686-w64-mingw32-gcc --version" says: i686-w64-mingw32-gcc (GCC) 4.6.3

> Cross-building on GNU/Linux is probably the cause of success: the
> problematic steps run on a Posix platform using Posix APIs, I presume?

Possibly.  Anyway, the point is that madsy built a working Guile with
thread support on Windows, and it seems to work, so your claim that
"Threads don't work in the MinGW build anyway" seems to be false.

>> > The only other way I know of is to bypass Posix-like functions like
>> > 'open', 'close', and 'dup', and go to the level of Win32 API, where a
>> > function that creates a child subprocess can accept handles for the
>> > child's standard I/O.
>> 
>> This sounds to me like the right approach in this case.
>
> OK.
>
>> > If you will be comfortable with a lot more Widows specific stuff, I
>> > can work on that.
>> 
>> Please do!
>
> OK, but please don't block this patch while you wait for me to do
> that.  It is certainly much better to have this functionality with
> restrictions than not at all.

I'd rather wait until we have a proper implementation that works with
multi-threaded programs.

     Thanks,
       Mark