Re: Bugfix for utf-8 XTerm/MinTTY and (set-input-meta-mode t)

[Max Mikhanosha]

On Tuesday, June 1st, 2021 at 12:51 PM, Eli Zaretskii <eliz@gnu.org> wrote:

> > Date: Tue, 01 Jun 2021 16:19:40 +0000
>
> > From: Max Mikhanosha max.mikhanosha@protonmail.com
> >
> > Both XTerm and MinTTY, when configured to send meta modifier as 8th bit 
> > while in utf-8 mode, will first add 8th bit, and then encode resulting 
> > character with utf-8. For example Meta-X is encoded as ?x+120 = #248 
> > codepoint, encoded as 0xc3,0xb8
>
> If these terminal emulators send M-x as 248 decimal, then how do they
> send the Latin character ø, whose codepoint is 248 decimal?

Exactly the same 0xc3,0xb8. Its point of the encode meta-key in the 8th bit 
mechanics, that you give up ability to type upper half of ASCII table (128-255) 
instead using them to indicate meta key.  Its a life-safer for those of us 
using vi, (on in Emacs case evil), otherwise sending meta keys as Esc <key> 
really interferes with vi mode where Esc key is super-special..  You can still 
insert these characters by doing quoted-insert, ie C-v M-x and it will insert ø

To illustrate my point, this patch restores the parity/functionality between 
non-utf-8 enabled and utf-8 enabled terminal

Before the patch, emacs is configured

non-utf8-xterm & emacs with (set-input-meta-mode nil), meta-x produces ø
non-utf8-xterm & emacs with (set-input-meta-mode t), meta-x produces M-x
utf8-xterm & emacs with (set-input-meta-mode nil), meta-x produces ø
utf8-xterm & emacs with (set-input-meta-mode t), meta-x produces weird garbage, 
C-v M-x produces weird garbage

After the patch

non-utf8-xterm & emacs with (set-input-meta-mode nil), meta-x produces ø
non-utf8-xterm & emacs with (set-input-meta-mode t), meta-x produces M-x, C-v 
M-x produces ø
utf8-xterm & emacs with (set-input-meta-mode nil), meta-x produces ø
utf8-xterm & emacs with (set-input-meta-mode t), meta-x produces M-x, C-v M-x 
produces ø

I also typed some stuff in Russian and nothing seem to be broke in UTF8 in 
general.    If there is a concern that this patch can break something, maybe 
one way forward is to rework it to add another meaning to meta mode flag, like 
(set-input-meta-mode 'utf8), which would trigger what my patch is doing.  But 
current behavior, where meta_key is checked before utf-8 decoding is obviously 
a bug, as it produces garbage input by treating utf-8 encoding itself as having 
meta modifiers.