Re: [vile] vile-9.6j.patch.gz

[Thomas Dickey]

On Sun, 30 Mar 2008, Thomas Dickey wrote:

> I see one problem (will fix...): when I cut/paste your example to create a 
> new file with either file-encoding=auto or utf-8, and saved it, I get an 
> iso-8859-1 file.

hmm - ignore that comment: I did see it written as iso-8859-1, but think
that was perhaps an error setting up one of the checks, cannot reproduce
it when I'm _looking_ for it.  (I ran it first on the installed copy, then
built a tracing version so I could verify that the codes were coming in
as UTF-8, and lost the trail around that point ;-)

> > > Without a specific setting, vile's reading characters according to
> > > the locale (with or without iconv), and applying them to the buffers
> > > according to _their_ file-encoding.
> >
> > % locale
> > LANG=en_US.UTF-8
> > LC_CTYPE=nl_NL.UTF8
> > LC_NUMERIC="en_US.UTF-8"
> > LC_TIME="en_US.UTF-8"
> > LC_COLLATE="en_US.UTF-8"
> > LC_MONETARY=nl_NL.UTF8
> > LC_MESSAGES="en_US.UTF-8"
> > LC_PAPER=nl_NL.UTF8
> > LC_NAME=nl_NL.UTF8
> > LC_ADDRESS=nl_NL.UTF8
> > LC_TELEPHONE=nl_NL.UTF8
> > LC_MEASUREMENT=nl_NL.UTF8
> > LC_IDENTIFICATION="en_US.UTF-8"
> > LC_ALL=
> >
> > > With xterm, I "just" use a meta-whatever to get upper-128 codes
> > > (which uxterm translates into UTF-8).  A quick check using vile's
> > > ^VxNN seems to be working.
> >
> > My xterm behaves oddly.. compose key usages seems to result in
> > no chars being inputted at all, but I can copy & paste UTF-8 chars
> > into the xterm (shell prompt).

I dimly recall some comment regarding gnome-terminal and compose keys.
google finds this, which indicates that it can do that, but isn't by
default:

https://lists.ubuntu.com/archives/ubuntu-users/2006-January/063131.html

However, this seems to say it's not:

https://bugs.launchpad.net/ubuntu/+source/xkeyboard-config/+bug/98945

(xterm uses a different set of libraries than gnome for this, so it'll
be broken by a different set of developers ;-)

> > Anyway, if I edit the file attached, both in xterm and gnome-terminal
> > with the above shown locales I see:

Using the same locale settings on my Debian/testing, I'm seeing the
text as you intended it (accented eoai, with dashes).  That's with
either uxterm (xterm displaying UTF-8 and corresponding fonts), or
with gnome-terminal (displaying UTF-8 - I suppose it always uses the
ISO-10646-1 fonts).

> > Accents: \?E9\?F3\?E1\?ED
> > Dashes: ??? ???
> >
> > (So the en- and em-dash (U2013, U2014) are shown correctly, but the vowels
> > with accents are not).

That sounds as if it's in the same general area as the fix I made for
pound sign.  The dashes are codes that cannot be mistaken for Latin-1.
Setting unicode-as-hex for your testfile, I see

Accents: \u00E9\u00F3\u00E1\u00ED
Dashes: \u2013 \u2014

If vile's running in an eight-bit locale encoding, it attempts to display
the eight-bit codes as a single-character.  So the same file in a 
en_US.ISO-8859-1 would show those accents as well - the dashes would 
always be as shown in the unicode-as-hex format above.

One oddity that I can see at the moment is that using :show-printable,
the second column using your locale settings is showing things like \?FA,
where the en_US.UTF-8 that I normally use is showing the Latin-1 code.
(I expect the latter).  I'll investigate that - it might be part of the
problem.

--
Thomas E. Dickey
http://invisible-island.net
ftp://invisible-island.net