Re: ms-terminal in terminfo.src

[Thomas Dickey]

On Mon, Aug 05, 2019 at 09:12:24PM +0000, Mike Griese wrote:
> Hey Thomas, Jurgen,
> 
> First off, let me first say THANK YOU so much for all the information you've 
> compiled on invisible-island.net. I cannot tell you how many times I've been 
> to that site over the last four years working on the VT emulation of the 
> Windows Console. It's been an invaluable resource.
> 
> I don't think I have any issues with the actual values in the entry.  That
> largely seems accurate for what we currently have support for.  I think what
> I'm more worried about is this part:
> 
> # The task manager shows this as "OpenConsole.exe", which differs
> # from the "Windows Command Processor" used for the command-prompt.

I see (I had misread the task manager's display supposing there was
some renaming of the regular command-prompt).

> # The settings dialog does not work (unless the end user expects to open
> # profiles.json in Visual Studio).  There is no documentation, of course.
> #
> # Testing via an ssh connection, using openssh:
> # - the program sets TERM to cygwin if the tab is set to PowerShell,
> #   and to xterm-256color if "Legacy".  However, in the latter, more tests
> #   fail in vttest, which does not pay attention to TERM.
> 
> I'm going to try and explain a bunch of related things here so bear with me.
> 
> First off, before the Windows Terminal, there was only ever one console
> program on Windows, and that was conhost.exe.  Conhost.exe is the program
> that's responsible for being the terminal for both cmd.exe ("Command Prompt")
> and powershell.exe. 
> 
> When we released the Windows Terminal as an open-source project this last
> April, we also released the source for conhost.exe with it.  However, when
> you build conhost.exe externally, it builds as "OpenConsole.exe".  The reason
> for this is unimportant, this is mostly just an artifact of an iterative
> design.  OpenConsole.exe is the _exact same thing_ as conhost.exe.
> 
> When you launch a console (commandline) application on Windows, the OS will
> automatically create a conhost.exe for that application, and list the shell
> as the parent application of the conhost.  However, when you launch
> OpenConsole.exe directly, we have some code that'll automatically launch
> cmd.exe on your behalf.  In that scenario, OpenConsole.exe ends up being the
> parent of cmd.exe, which is why OpenConsole shows up differently from
> conhost.exe in the task manager.
> 
> I'd also be wary of using the openssh ssh.exe that ships inbox with Windows
> to run VT tests.  In my experience their implementation has been less than
> perfect for whatever reason.  We've gotten our fair share of bugs filed on us
> that are actually bugs in their emulation layer.  Running WSL directly within
> conhost works _great_ as a test bed for running vttest et.  al, and I've
> found the ssh that's provided with WSL distros to be more reliable. 

thanks - I'll look into that
 
> So far, I've only really been talking about conhost.exe, the Console.  The
> Windows Terminal is actually an even more complicated story.
> 
> Conhost.exe is not _only_ responsible for being the terminal window, but it's
> also responsible for being the Console API server.  The Console API on
> Windows is composed of some 70+ APIs, some of which make sense as VT
> sequences and others that are totally insane.  The Console API also includes
> a lot of features that would probably make more sense as a `readline` like
> library, rather than an integral part of the console. 
> 
> Rather than force terminal authors to deal with the quirks of the Console
> API, we instead created Conpty, which acts as a translation layer from the
> Console API to VT sequences that would be understood by any existing
> terminal.  At the core on conpty is conhost.exe, still acting as the Console
> API server, but now it's _rendering_ its state to a stream of characters and
> VT sequences.  These characters are emitted to another pipe which a terminal
> program can read to recreate the state of the console buffer. 
> 
> The Windows Terminal actually then uses the conpty API and the stream of
> chars conpty generates, not the output of the application directly.  The
> Windows Terminal is actually a pretty weak terminal emulator all by itself,
> because conhost.exe is doing most of the heavy lifting, and reducing the
> number of sequences the Terminal needs to know about to like, 10 total.  All
> of this _is_ supposed to be transparent to the end user, but many of the bugs
> in the Windows Terminal's "terminal emulation" are actually due to bugs in
> the conpty layer, not necessarily conhost's handling of those sequences.  For
> example, mouse input works pretty well in conhost.exe, but conpty isn't quite
> yet equipped to handle it. 
> 
> Neither conhost nor Windows Terminal are setting TERM, so I'd presume that
> the TERM value mentioned is being set by openssh.  I know that WSL will set
> TERM to xterm-256color.  If we wanted to be _technically_ correct, then we'd
> have separate entries for both conhost and the Windows Terminal.  Though, due
> to the aforementioned translation being done by conpty, this could become
> confusing very quickly.  If we leave anything in for the Windows Terminal,
> I'd want to make sure it was definitely marked **Experimental** or
> **Preview** or something to that effect.
> 
> The settings UI is not done yet - that's correct.  When a user tries to open
> the settings, we'll try to open the user's configured .json file editor. 
> Visual Studio does override this on install, but the user is free to use
> whatever editor they like.  As of v0.3, we'll open notepad if they don't have
> a .json editor installed.  We're still adding more documentation, but you can
> find the start of the documentation [here](
> https://github.com/microsoft/terminal/blob/master/doc/user-docs/index.md).
> 
> It was also mentioned that changing the color palette is tricky - That should
> be a lot better nowadays actually.  We released the [ColorTool](
> https://github.com/Microsoft/Terminal/tree/master/src/tools/ColorTool) a few
> years ago, which should make changing the colors much easier.  Additionally,
> we support OSC 4 (and similar sequences) for changing the palette at runtime.
> 
> Do you actually get bug reports for ncurses for users using conhost.exe
> and/or the Windows Terminal?  I'd certainly be curious to see those - our VT

sure - anyone using ssh is likely to notice problems or cygwin, WSL, etc.
But people aren't very good about reporting _here_ (or to me),
but tend to just complain --e.g., StackOverflow-- (so I look for them).

> emulation is getting better every release, but it still has a long way to go. 
> Getting more bug reports would help us be able to track the important things
> we need to work on next.  The list of diffs you've provided between
> `ms-terminal` and `xterm-256color` are a super helpful place to start next 😊
> 
> 
> Let us know if there's anything we can do to help here!
> 
> Mike Griese
> 
> 
> 
> -----Original Message-----
> From: Thomas Dickey <address@hidden> 
> Sent: Friday, August 2, 2019 3:52 PM
> To: Mike Griese <address@hidden>
> Cc: address@hidden; Windows Command Line Team <address@hidden>
> Subject: Re: ms-terminal in terminfo.src
> 
> On Fri, Aug 02, 2019 at 12:59:42PM +0000, Mike Griese via Bug-ncurses wrote:
> > To Whom It May Concern,
> > 
> > My name is Mike Griese, I work on the Windows Terminal Team.  It's 
> > come to our attention that there's a new 
> > ms-terminal<https://invisible-island.net/ncurses/terminfo.src.html#tic
> > -ms-terminal> in terminfo.src, though I'm fairly certain no one in our 
> > organization submitted that entry.  It looks to me like the entry was 
> > submitted in response to our announcement of the Windows Terminal.  
> > However there looks to be some problems with the listing, particularly 
> > in the comment with it.  Is there any way we could at very least 
> > update the comment to be more accurate?
> 
> Actually the comment's accurate.  It says what version was examined,
> 
>       # Windows 10 1903
>       # Version 0.2.1715.0
> 
> and lists several issues which led to omitting various features of the 
> "xterm-256color" terminal description to give a terminal description which 
> would work properly with the Windows Terminal.
> 
> If someone uses that version of Windows Terminal with TERM=xterm-256color, 
> I'll be seeing bug reports and negative comments about ncurses.
> 
> (If there's a newer released version of Windows Terminal, I can update).
> 
> You might find these helpful:
> 
> https://invisible-island.net/ncurses/tack/
> https://invisible-island.net/vttest/
> 
> --
> Thomas E. Dickey <address@hidden> https://invisible-island.net 
> ftp://ftp.invisible-island.net

-- 
Thomas E. Dickey <address@hidden>
https://invisible-island.net
ftp://ftp.invisible-island.net

signature.asc
Description: PGP signature