bug#64420: string-width of … is 2 in CJK environments

[Dmitry Gutov]

On 11/07/2023 14:48, Eli Zaretskii wrote:
> > Date: Tue, 11 Jul 2023 05:23:03 +0300
> > Cc: 64420@debbugs.gnu.org
> > From: Dmitry Gutov <dmitry@gutov.dev>
> >
> > On 02/07/2023 16:43, Eli Zaretskii wrote:
> > > > Since the overlay-based popup is used on both GUI and Terminal frames,
> > > > are you suggesting I define my own string-width like this?
> > > >
> > > > (defun company--string-width (str)
> > > >      (if (display-graphic-p)
> > > >          (ceiling (/ (string-pixel-width str)
> > > >                      (float (default-font-width))))
> > > >        (string-width str)))
> > > Yes, definitely.  (Actually, display-multi-font-p is better than
> > > display-graphic-p, but in practice they will return the same value.)
> >
> > Regarding this approach, though: it seems to fail in my terminal Emacs.
>
> string-pixel-width is useless on TTY frames, because Emacs cannot
> access the metrics of the characters on those frames.  In those cases
> string-pixel-width falls back to use char-width, and you get the same
> result.

I guess that's the best we can do. This seems to work okay with most 
double-width characters, as long as the reported metrics match what 
happens on display.

And according to your explanation, we could probably drop the 
display-graphic-p check since both branches result in the same value on 
terminal (right?).

> > Meaning, when I'm testing the feature using 'emacs -nw' (inside e.g.
> > gnome-terminal), both (string-pixel-width "…") and (string-width "…")
> > return 2. Whereas the character on display looks 1-character wide even
> > there.
>
> Once again, the assumption behind this "feature" of the CJK
> language-environments is that whoever uses those environments has the
> terminal emulators configured to use fonts where "…" and its ilk have
> double size.  Of course, if you just switch language-environment on a
> system that is otherwise configured for non-CJK locale, the terminal
> emulator fonts will not magically change, and you get what you see.

Does "…" actually have double width in some of their fonts?

This report stems from an issue opened on Github for company-mode (see 
the first message) from somebody who as I understand hails from one of 
those countries (I haven't clarified exactly), and they apparently have 
to work with the "Chinese-BIG5" language environment.

Are you saying that they misconfigured their system somehow, e.g. that 
Chinese-BIG5 is expected to be used with a certain set of default system 
fonts which have "…" at double width?

> > More than that, moving the cursor close to that character with C-f or
> > C-b creates odd effects like the cursor jumping one position to the
> > left, or a char being rendered twice at a certain position on the same
> > line to the right of it (after I move the cursor there past the … char),
>
> Yes, because we lie to the display engine about the character width.
>
> If you worry that something in your package might not work well for
> some users due to this issue, how about giving them a user-level
> option to change the char-width of this character to 1?

It's been suggested that we alter char-width-table dynamically too, as 
one option. I was just hoping to clarify that we don't carry an 
erroneous entry for this particular character.

If we did, it would be an easier solution for me to direct the users to 
the fix in Emacs 29/30, and delay the rollout of the new popup rendering 
feature a little bit. It will need a fair bit of testing period given 
the nature of the change.

Further, string-pixel-width and buffer-text-pixel-size have only been 
added in Emacs 29. Any chance you know some replacement I could use to 
backport the functionality to work in Emacs 25 or 26? 
buffer-text-pixel-size is defined in C.