bug#10867: 26.3; XIM preedit/status font handling

[Yichao Yu]

This is a really old bug but it is finally clear recently what is causing it.

## Background and observed issues.

1. Since at least 8 years ago when this bug was first reported, it was
observed that sometimes emacs cannot use XIM unless set to a CJK
locale, e.g. zh_CN.UTF-8. This is not always necessary, and might not
always work and it was also unclear what caused it since nothing seems
to have hard coded anything locale dependent in either emacs or xlib
and other programs using xim also seems fine...

2. It was also know (probably more recent) that it is necessary to
install certain font to get XIM work with emacs since otherwise some
XIM related font lookup fails and disables XIM.

## Root cause
It turns out that the two issues are closely related. The full story
is roughly like this,

1. Emacs calls `xic_create_xfontset` in `create_frame_xic`. This
function try to figure out a list of fonts based on some conditions.
If no font is found at this state, it causes the second issue that
requires installation of some fonts.
2. The return value of this is then passed in as the `fontSet`
(XNFontSet) attribute for preedit and status within
`create_frame_xic`. However, if there are too many font matches, it'll
fail in the `XimType_XFontSet` case within `_XimValueToAttribute`
since it can overflow the 2048 byte buffer that xlib uses. (the buffer
size available when it decodes the XFontSet measured locally is ~ 1k).
This also causes the input method initialization to fail and this is
the cause of the first issue. The effect of setting the locale is
actually simply on the number of font matches. Most users seem to have
more fonts matching an english locale than a CJK one which is why
setting to a CJK locale avoids the overflow. For users that have
fewer/different (english) fonts installed there could also be fewer
matches allowing XIM to work.

## What other programs do
* xterm seems to use "fixed" to lookup font
* qt4 https://dreamswork.github.io/qt4/qximinputcontext__x11_8cpp_source.html
AFAICT uses different fonts?
* gtk I don't actually see where it set fonts...
@Xuetian Weng dug deeper into this

## Fix
I'm not sure what's the correct one. It seems that setting the fontSet
is necessary although I'm not sure how/if gtk get away with it.
According to @Xuetian Weng virtually no one use this setting anymore
so setting a dummy value to comfort xlib might be good enough? It
would avoid both issues at the same time.

-------------------

In GNU Emacs 26.3 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.24.21)
 of 2020-07-29 built on yyc.yyc-arch.org
Repository revision: d9eea1ad1bca16406a6d4c9bd0daa57a9d0a531f
System Description: Arch Linux

Recent messages:
For information about GNU Emacs and the GNU system, type C-h C-a.
Making completion list...

Configured using:
 'configure --prefix=/usr --sysconfdir=/etc --libexecdir=/usr/lib
 --localstatedir=/var --with-x-toolkit=gtk3 --with-xft --with-wide-int
 --with-modules 'CFLAGS=-march=x86-64 -mtune=generic -O2 -pipe -fno-plt
 -g -fvar-tracking-assignments
 
-fdebug-prefix-map=/home/yuyichao/projects/arch-pkg/pkg/tmp/emacs/repos/extra-x86_64/src=/usr/src/debug
 -ggdb3 -O0 -DDEBUG_XIC_FONTSET=1' CPPFLAGS=-D_FORTIFY_SOURCE=2
 LDFLAGS=-Wl,-O1,--sort-common,--as-needed,-z,relro,-z,now'

Configured features:
XPM JPEG TIFF GIF PNG RSVG IMAGEMAGICK SOUND GPM DBUS GSETTINGS GLIB
NOTIFY ACL GNUTLS LIBXML2 FREETYPE M17N_FLT LIBOTF XFT ZLIB
TOOLKIT_SCROLL_BARS GTK3 X11 XDBE XIM MODULES THREADS LIBSYSTEMD LCMS2

Important settings:
  value of $LANG: zh_CN.UTF-8
  value of $XMODIFIERS: @im=fcitx
  locale-coding-system: utf-8-unix

Major mode: Fundamental

Minor modes in effect:
  tooltip-mode: t
  global-eldoc-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  buffer-read-only: t
  line-number-mode: t
  transient-mark-mode: t

Load-path shadows:
/usr/share/emacs/site-lisp/org/ox hides /usr/share/emacs/26.3/lisp/org/ox
/usr/share/emacs/site-lisp/org/ox-texinfo hides
/usr/share/emacs/26.3/lisp/org/ox-texinfo
/usr/share/emacs/site-lisp/org/ox-publish hides
/usr/share/emacs/26.3/lisp/org/ox-publish
/usr/share/emacs/site-lisp/org/ox-org hides
/usr/share/emacs/26.3/lisp/org/ox-org
/usr/share/emacs/site-lisp/org/ox-odt hides
/usr/share/emacs/26.3/lisp/org/ox-odt
/usr/share/emacs/site-lisp/org/ox-md hides /usr/share/emacs/26.3/lisp/org/ox-md
/usr/share/emacs/site-lisp/org/ox-man hides
/usr/share/emacs/26.3/lisp/org/ox-man
/usr/share/emacs/site-lisp/org/ox-latex hides
/usr/share/emacs/26.3/lisp/org/ox-latex
/usr/share/emacs/site-lisp/org/ox-icalendar hides
/usr/share/emacs/26.3/lisp/org/ox-icalendar
/usr/share/emacs/site-lisp/org/ox-html hides
/usr/share/emacs/26.3/lisp/org/ox-html
/usr/share/emacs/site-lisp/org/ox-beamer hides
/usr/share/emacs/26.3/lisp/org/ox-beamer
/usr/share/emacs/site-lisp/org/ox-ascii hides
/usr/share/emacs/26.3/lisp/org/ox-ascii
/usr/share/emacs/site-lisp/org/org hides /usr/share/emacs/26.3/lisp/org/org
/usr/share/emacs/site-lisp/org/org-version hides
/usr/share/emacs/26.3/lisp/org/org-version
/usr/share/emacs/site-lisp/org/org-timer hides
/usr/share/emacs/26.3/lisp/org/org-timer
/usr/share/emacs/site-lisp/org/org-table hides
/usr/share/emacs/26.3/lisp/org/org-table
/usr/share/emacs/site-lisp/org/org-src hides
/usr/share/emacs/26.3/lisp/org/org-src
/usr/share/emacs/site-lisp/org/org-protocol hides
/usr/share/emacs/26.3/lisp/org/org-protocol
/usr/share/emacs/site-lisp/org/org-plot hides
/usr/share/emacs/26.3/lisp/org/org-plot
/usr/share/emacs/site-lisp/org/org-pcomplete hides
/usr/share/emacs/26.3/lisp/org/org-pcomplete
/usr/share/emacs/site-lisp/org/org-mouse hides
/usr/share/emacs/26.3/lisp/org/org-mouse
/usr/share/emacs/site-lisp/org/org-mobile hides
/usr/share/emacs/26.3/lisp/org/org-mobile
/usr/share/emacs/site-lisp/org/org-macs hides
/usr/share/emacs/26.3/lisp/org/org-macs
/usr/share/emacs/site-lisp/org/org-macro hides
/usr/share/emacs/26.3/lisp/org/org-macro
/usr/share/emacs/site-lisp/org/org-loaddefs hides
/usr/share/emacs/26.3/lisp/org/org-loaddefs
/usr/share/emacs/site-lisp/org/org-list hides
/usr/share/emacs/26.3/lisp/org/org-list
/usr/share/emacs/site-lisp/org/org-lint hides
/usr/share/emacs/26.3/lisp/org/org-lint
/usr/share/emacs/site-lisp/org/org-install hides
/usr/share/emacs/26.3/lisp/org/org-install
/usr/share/emacs/site-lisp/org/org-inlinetask hides
/usr/share/emacs/26.3/lisp/org/org-inlinetask
/usr/share/emacs/site-lisp/org/org-indent hides
/usr/share/emacs/26.3/lisp/org/org-indent
/usr/share/emacs/site-lisp/org/org-id hides
/usr/share/emacs/26.3/lisp/org/org-id
/usr/share/emacs/site-lisp/org/org-habit hides
/usr/share/emacs/26.3/lisp/org/org-habit
/usr/share/emacs/site-lisp/org/org-footnote hides
/usr/share/emacs/26.3/lisp/org/org-footnote
/usr/share/emacs/site-lisp/org/org-feed hides
/usr/share/emacs/26.3/lisp/org/org-feed
/usr/share/emacs/site-lisp/org/org-faces hides
/usr/share/emacs/26.3/lisp/org/org-faces
/usr/share/emacs/site-lisp/org/org-entities hides
/usr/share/emacs/26.3/lisp/org/org-entities
/usr/share/emacs/site-lisp/org/org-element hides
/usr/share/emacs/26.3/lisp/org/org-element
/usr/share/emacs/site-lisp/org/org-duration hides
/usr/share/emacs/26.3/lisp/org/org-duration
/usr/share/emacs/site-lisp/org/org-datetree hides
/usr/share/emacs/26.3/lisp/org/org-datetree
/usr/share/emacs/site-lisp/org/org-ctags hides
/usr/share/emacs/26.3/lisp/org/org-ctags
/usr/share/emacs/site-lisp/org/org-crypt hides
/usr/share/emacs/26.3/lisp/org/org-crypt
/usr/share/emacs/site-lisp/org/org-compat hides
/usr/share/emacs/26.3/lisp/org/org-compat
/usr/share/emacs/site-lisp/org/org-colview hides
/usr/share/emacs/26.3/lisp/org/org-colview
/usr/share/emacs/site-lisp/org/org-clock hides
/usr/share/emacs/26.3/lisp/org/org-clock
/usr/share/emacs/site-lisp/org/org-capture hides
/usr/share/emacs/26.3/lisp/org/org-capture
/usr/share/emacs/site-lisp/org/org-attach hides
/usr/share/emacs/26.3/lisp/org/org-attach
/usr/share/emacs/site-lisp/org/org-archive hides
/usr/share/emacs/26.3/lisp/org/org-archive
/usr/share/emacs/site-lisp/org/org-agenda hides
/usr/share/emacs/26.3/lisp/org/org-agenda
/usr/share/emacs/site-lisp/org/ob hides /usr/share/emacs/26.3/lisp/org/ob
/usr/share/emacs/site-lisp/org/ob-vala hides
/usr/share/emacs/26.3/lisp/org/ob-vala
/usr/share/emacs/site-lisp/org/ob-tangle hides
/usr/share/emacs/26.3/lisp/org/ob-tangle
/usr/share/emacs/site-lisp/org/ob-table hides
/usr/share/emacs/26.3/lisp/org/ob-table
/usr/share/emacs/site-lisp/org/ob-stan hides
/usr/share/emacs/26.3/lisp/org/ob-stan
/usr/share/emacs/site-lisp/org/ob-sqlite hides
/usr/share/emacs/26.3/lisp/org/ob-sqlite
/usr/share/emacs/site-lisp/org/ob-sql hides
/usr/share/emacs/26.3/lisp/org/ob-sql
/usr/share/emacs/site-lisp/org/ob-shen hides
/usr/share/emacs/26.3/lisp/org/ob-shen
/usr/share/emacs/site-lisp/org/ob-shell hides
/usr/share/emacs/26.3/lisp/org/ob-shell
/usr/share/emacs/site-lisp/org/ob-sed hides
/usr/share/emacs/26.3/lisp/org/ob-sed
/usr/share/emacs/site-lisp/org/ob-screen hides
/usr/share/emacs/26.3/lisp/org/ob-screen
/usr/share/emacs/site-lisp/org/ob-scheme hides
/usr/share/emacs/26.3/lisp/org/ob-scheme
/usr/share/emacs/site-lisp/org/ob-sass hides
/usr/share/emacs/26.3/lisp/org/ob-sass
/usr/share/emacs/site-lisp/org/ob-ruby hides
/usr/share/emacs/26.3/lisp/org/ob-ruby
/usr/share/emacs/site-lisp/org/ob-ref hides
/usr/share/emacs/26.3/lisp/org/ob-ref
/usr/share/emacs/site-lisp/org/ob-python hides
/usr/share/emacs/26.3/lisp/org/ob-python
/usr/share/emacs/site-lisp/org/ob-processing hides
/usr/share/emacs/26.3/lisp/org/ob-processing
/usr/share/emacs/site-lisp/org/ob-plantuml hides
/usr/share/emacs/26.3/lisp/org/ob-plantuml
/usr/share/emacs/site-lisp/org/ob-picolisp hides
/usr/share/emacs/26.3/lisp/org/ob-picolisp
/usr/share/emacs/site-lisp/org/ob-perl hides
/usr/share/emacs/26.3/lisp/org/ob-perl
/usr/share/emacs/site-lisp/org/ob-org hides
/usr/share/emacs/26.3/lisp/org/ob-org
/usr/share/emacs/site-lisp/org/ob-octave hides
/usr/share/emacs/26.3/lisp/org/ob-octave
/usr/share/emacs/site-lisp/org/ob-ocaml hides
/usr/share/emacs/26.3/lisp/org/ob-ocaml
/usr/share/emacs/site-lisp/org/ob-mscgen hides
/usr/share/emacs/26.3/lisp/org/ob-mscgen
/usr/share/emacs/site-lisp/org/ob-maxima hides
/usr/share/emacs/26.3/lisp/org/ob-maxima
/usr/share/emacs/site-lisp/org/ob-matlab hides
/usr/share/emacs/26.3/lisp/org/ob-matlab
/usr/share/emacs/site-lisp/org/ob-makefile hides
/usr/share/emacs/26.3/lisp/org/ob-makefile
/usr/share/emacs/site-lisp/org/ob-lua hides
/usr/share/emacs/26.3/lisp/org/ob-lua
/usr/share/emacs/site-lisp/org/ob-lob hides
/usr/share/emacs/26.3/lisp/org/ob-lob
/usr/share/emacs/site-lisp/org/ob-lisp hides
/usr/share/emacs/26.3/lisp/org/ob-lisp
/usr/share/emacs/site-lisp/org/ob-lilypond hides
/usr/share/emacs/26.3/lisp/org/ob-lilypond
/usr/share/emacs/site-lisp/org/ob-ledger hides
/usr/share/emacs/26.3/lisp/org/ob-ledger
/usr/share/emacs/site-lisp/org/ob-latex hides
/usr/share/emacs/26.3/lisp/org/ob-latex
/usr/share/emacs/site-lisp/org/ob-js hides /usr/share/emacs/26.3/lisp/org/ob-js
/usr/share/emacs/site-lisp/org/ob-java hides
/usr/share/emacs/26.3/lisp/org/ob-java
/usr/share/emacs/site-lisp/org/ob-io hides /usr/share/emacs/26.3/lisp/org/ob-io
/usr/share/emacs/site-lisp/org/ob-hledger hides
/usr/share/emacs/26.3/lisp/org/ob-hledger
/usr/share/emacs/site-lisp/org/ob-haskell hides
/usr/share/emacs/26.3/lisp/org/ob-haskell
/usr/share/emacs/site-lisp/org/ob-groovy hides
/usr/share/emacs/26.3/lisp/org/ob-groovy
/usr/share/emacs/site-lisp/org/ob-gnuplot hides
/usr/share/emacs/26.3/lisp/org/ob-gnuplot
/usr/share/emacs/site-lisp/org/ob-fortran hides
/usr/share/emacs/26.3/lisp/org/ob-fortran
/usr/share/emacs/site-lisp/org/ob-forth hides
/usr/share/emacs/26.3/lisp/org/ob-forth
/usr/share/emacs/site-lisp/org/ob-exp hides
/usr/share/emacs/26.3/lisp/org/ob-exp
/usr/share/emacs/site-lisp/org/ob-eval hides
/usr/share/emacs/26.3/lisp/org/ob-eval
/usr/share/emacs/site-lisp/org/ob-emacs-lisp hides
/usr/share/emacs/26.3/lisp/org/ob-emacs-lisp
/usr/share/emacs/site-lisp/org/ob-ebnf hides
/usr/share/emacs/26.3/lisp/org/ob-ebnf
/usr/share/emacs/site-lisp/org/ob-dot hides
/usr/share/emacs/26.3/lisp/org/ob-dot
/usr/share/emacs/site-lisp/org/ob-ditaa hides
/usr/share/emacs/26.3/lisp/org/ob-ditaa
/usr/share/emacs/site-lisp/org/ob-css hides
/usr/share/emacs/26.3/lisp/org/ob-css
/usr/share/emacs/site-lisp/org/ob-core hides
/usr/share/emacs/26.3/lisp/org/ob-core
/usr/share/emacs/site-lisp/org/ob-coq hides
/usr/share/emacs/26.3/lisp/org/ob-coq
/usr/share/emacs/site-lisp/org/ob-comint hides
/usr/share/emacs/26.3/lisp/org/ob-comint
/usr/share/emacs/site-lisp/org/ob-clojure hides
/usr/share/emacs/26.3/lisp/org/ob-clojure
/usr/share/emacs/site-lisp/org/ob-calc hides
/usr/share/emacs/26.3/lisp/org/ob-calc
/usr/share/emacs/site-lisp/org/ob-awk hides
/usr/share/emacs/26.3/lisp/org/ob-awk
/usr/share/emacs/site-lisp/org/ob-asymptote hides
/usr/share/emacs/26.3/lisp/org/ob-asymptote
/usr/share/emacs/site-lisp/org/ob-abc hides
/usr/share/emacs/26.3/lisp/org/ob-abc
/usr/share/emacs/site-lisp/org/ob-R hides /usr/share/emacs/26.3/lisp/org/ob-R
/usr/share/emacs/site-lisp/org/ob-J hides /usr/share/emacs/26.3/lisp/org/ob-J
/usr/share/emacs/site-lisp/org/ob-C hides /usr/share/emacs/26.3/lisp/org/ob-C

Features:
(shadow sort mail-extr emacsbug message rmc puny seq byte-opt gv
bytecomp byte-compile cconv cl-loaddefs cl-lib dired dired-loaddefs
format-spec rfc822 mml easymenu mml-sec password-cache epa derived epg
epg-config gnus-util rmail rmail-loaddefs mm-decode mm-bodies mm-encode
mail-parse rfc2231 mailabbrev gmm-utils mailheader sendmail rfc2047
rfc2045 ietf-drums mm-util mail-prsvr mail-utils term/xterm xterm
time-date elec-pair mule-util china-util tooltip eldoc electric uniquify
ediff-hook vc-hooks lisp-float-type mwheel term/x-win x-win
term/common-win x-dnd tool-bar dnd fontset image regexp-opt fringe
tabulated-list replace newcomment text-mode elisp-mode lisp-mode
prog-mode register page menu-bar rfn-eshadow isearch timer select
scroll-bar mouse jit-lock font-lock syntax facemenu font-core
term/tty-colors frame cl-generic cham georgian utf-8-lang misc-lang
vietnamese tibetan thai tai-viet lao korean japanese eucjp-ms cp51932
hebrew greek romanian slovak czech european ethiopic indian cyrillic
chinese composite charscript charprop case-table epa-hook jka-cmpr-hook
help simple abbrev obarray minibuffer cl-preloaded nadvice loaddefs
button faces cus-face macroexp files text-properties overlay sha1 md5
base64 format env code-pages mule custom widget hashtable-print-readable
backquote threads dbusbind inotify lcms2 dynamic-setting
system-font-setting font-render-setting move-toolbar gtk x-toolkit x
multi-tty make-network-process emacs)

Memory information:
((conses 16 100413 6755)
 (symbols 48 20515 1)
 (miscs 40 73 105)
 (strings 32 28736 1427)
 (string-bytes 1 750294)
 (vectors 16 13057)
 (vector-slots 8 514213 20152)
 (floats 8 54 437)
 (intervals 56 232 0)
 (buffers 992 13))