Re: [groff] devutf8 on Windows

[Eli Zaretskii]

> From: Jeff Conrad <address@hidden>
> Date: Sun, 24 Feb 2019 01:04:22 +0000
> 
> I’m getting strange behavior with devutf8 on Windows.
> 
> >type quot.txt
> .pl 1
> \(oqquote\(cq minus: \(mi /f/: /\(f/
> 
> With the code page set to 65001 (UTF-8), from cmd,

What version of Windows is that?

> >groff -Tutf8 quot.txt
> ‘quote’ minus: − /f/: /⁄
> 
> The output looks as if it’s being interpreted as code page 1252. If the
> output is redirected, things look fine, e.g.,
> 
> >groff -Tutf8 quot.txt | more.com
> ‘quote’ minus: − /f/: /⁄

Which version of 'more' is that?  The one which comes with Windows 7
fails with "not enough memory".  I needed to use a ported Less to have
this displayed.

> or just
> 
> >groff -Tutf8 quot.txt >CON,
> ‘quote’ minus: − /f/: /⁄
> 
> the output is fine.
> 
> I get the same results with the MKS Korn shell.  If the code page is set
> to 1252 rather than 65001, results are as expected—rendered as code
> page 1252.

That's a known issue, see below.

> If I put tty.cpp’s put_char() in a simple program (compiled with
> Visual Studio 2015) and feed it Unicode values, the output is what I
> expect—properly rendered UTF-8.

How did you "feed it Unicode values", exactly?  And what was the
simple program you used?

> Any ideas as to what could be happening?

It is a known problem with the Windows console: you cannot reliably
write UTF-8 encoded text to it using the ANSI/Posix emulation
functions like 'write', 'printf', and their C++ equivalents, and
expect the correct display, even if you set the codepage to 65001.
You need to use Unicode (a.k.a. "wide") APIs instead, like WriteFileW
etc., and you need to feed them text converted into UTF-16.

This is not a Groff problem, the problem is with the Windows console
device.

I heard rumors that Windows 10 is better in this regard, but I don't
have a Windows 10 box around with Groff on it, so I cannot test that.

> I’m running the ezwinports Win32 binary 1.22.3 (the 1.22.4 grotty binary
> does the same thing).

ezwinports has Groff 1.22.4 since almost 2 weeks ago.