Re: ACS_VLINE characters shifted in UTF-8 xterm

[Thomas Dickey]

On Sun, Apr 05, 2009 at 01:35:49PM +0200, Sebastian Kayser wrote:
> Hi,
> 
> i have an ncurses-based application that draws a vertical line via
> ACS_VLINEs to the screen. When i run this application in an UTF-8 xterm
> (xterm-243) the ACS_VLINEs come out as a kind of a right-shifted
> staircase. When i set NCURSES_NO_UTF8_ACS=1 or when i run the
> application in dtterm the vertical line looks fine.
> 
> The application in question is mcabber [1], but i can recreate this
> issue with a few lines of code [2]. It doesn't matter whether i link
> against a regular ncurses or a --enable-widec one.
> 
> I have tried to wrap my head around this and read the relevant ncurses
> FAQ items. From what i understand xterm is capable of interpreting the
> ACS form characters just fine even in UTF-8 mode (so no need to set
> NCURSES_NO_UTF8_ACS). I have truss'ed the application to see how the
> vertical line gets written to the terminal.
> 
>   1B [ H     [ s t a t u s ]                          0E x0F1B [ m
>   1B [ 1 B\b0E x0F1B [ m1B [ 1 B\b0E x0F1B [ m1B [ 1 B\b0E x0F1B [
>    m1B [ 1 B\b0E x0F1B [ m1B [ 1 B\b0E x0F1B [ m1B [ 1 B\b0E x0F1B
>    [ m1B [ 1 B\b0E x0F1B [ m1B [ 1 B\b0E x0F1B [ m1B [ 1 B\b0E x0F
>   1B [ m1B [ 1 B\b0E x0F1B [ m1B [ 1 B\b0E x0F1B [ m1B [ 1 B\b0E x
> 
> So it is basically a sequence of "smacs, x, rmacs, sgr0, cud, \b"
> sequences to draw the line. When i manually echo ACS_VLINE characters to
> the terminal i can see that they consume two columns instead of one and
> i suppose that's why the \b isn't sufficient to move completely backwards.

Here's a guess:

Both ncurses and xterm check the widths of characters using wcwidth.

However, ncurses doesn't check the ACS_xxx widths (lots of things
would break if they're allowed to be something other than 1).

xterm may internally translate the VT100-style line-drawing to Unicode
values (see charsets.c around line 310), and if wcwidth returns an
unexpected value for that, it could produce the sort of effect
you're describing.
 
> $ echo -e 'X(B X)0 \x0e x x x \x0f' | tr -d ' ' | tr 'X' '\033'
> 
> Pasting those vertical lines into od tells me that they are "BOX
> DRAWINGS LIGHT VERTICAL". What exactly is responsible for printing them
> instead of just "VERTICAL LINE" characters?

Sounds like xterm (it'll store the actual character which is used
for display in the cells which are used for selection).
 
> $ cat | od -tx1
> ???
> 0000000 e2 94 82 e2 94 82 e2 94 82 0a
> 0000012
> 
> Amending the sample application so that it writes a second vertical line
> 10 columns next to the first one (see the commented out line in the
> sample code) makes ncurses only use CUP sequences for positioning and
> the vertical lines look fine then (although i would assume that
> character positioning after one such vertical line would still be shifted).
> 
> I am using Solaris 10 in case that matters, LC_CTYPE is set to
> en_US.UTF-8. It seems as if a missing only a small part of the puzzle.
> What's wrong with those vertical line characters and my UTF-8 xterm?

Solaris locale support seems to have some quirks (I could digress).

I'd try setting the mkWidth resource, which _should_ tell xterm to
use its built-in locale table.

There's a command-line option (-mk_width).

-- 
Thomas E. Dickey <address@hidden>
http://invisible-island.net
ftp://invisible-island.net

signature.asc
Description: Digital signature