bug#69598: 29.2; colour support based on $TERM value not terminfo database

[Eli Zaretskii]

> From: chohag@jtan.com
> cc: chohag@jtan.com, 69598@debbugs.gnu.org
> Comments: In-reply-to Eli Zaretskii <eliz@gnu.org>
>    message dated "Fri, 08 Mar 2024 09:11:03 +0200."
> Date: Fri, 08 Mar 2024 11:36:18 +0000
> 
> Eli Zaretskii writes:
> > > From: chohag@jtan.com
> > > cc: chohag@jtan.com, 69598@debbugs.gnu.org
> > > Comments: In-reply-to Eli Zaretskii <eliz@gnu.org>
> > >    message dated "Thu, 07 Mar 2024 22:10:30 +0200."
> > > Date: Thu, 07 Mar 2024 21:45:44 +0000
> >
> > Thanks.  These results AFAIU indicate that there's no problem with the
> > terminfo level: the number of colors known to term.c (printed as max:)
> > is the same no matter how you start Emacs.  Do you agree with this
> > conclusion?
> 
> That does appear to be the case. In fact the number of colours is
> set by 'tty->TN_max_colors = tgetnum ("Co")' and then set again
> when the RGB or other direct-colour capability is detected.

That's correct, but why is that an issue?  What I saw in your
debugging print-outs is that the value of tty->TN_max_colors was the
same in all of the 3 configurations you tried.  Since your changes
printed this value after the code that determines how many colors are
supported, I concluded that terminal initialization computed the same
number of colors in all 3 cases.  Did I miss something?

> Unfortunately the information from terminfo is sometimes ignored.

Why "unfortunately", if the result is the correct number of colors?

> > I think the problem is that if the terminal's name doesn't begin with
> > "xterm", Emacs does not load lisp/term/xterm.el, which defines the
> > colors that will be available.  So please copy lisp/term/xterm.el to a
> > file named lisp/term/notworking.el, and then start Emacs with
> > TERM=notworking and see if that solves the problem.  That is, arrange
> > for a lisp/term/NAME.el file to be a copy of xterm.el when you invoke
> > Emacs with TERM=NAME.
> 
> If we're at the point where the terminal's name matters one jot,
> that is to say when the value of $TERM is used for _anything_ other
> than selecting a terminfo entry, that is a bug.

It is not necessarily a bug.  Emacs needs to be able to convert
X-style color names to escape sequences it should send to the terminal
to produce a similar color.  That conversion is specific to the
terminal, so Emacs relies on the name of the terminal to set up the
conversion.

We could have a new feature whereby, if terminfo redirected to a
different entry, Emacs would load the lisp/term file of the redirected
entry.  But that'd be a new feature that AFAIU never existed in Emacs.

In any case, I suggest to delay the discussion of what should be the
solution to the issue until we understand the issue completely.
AFAIU, we are not there yet, see above.

> I don't know how emacs finds files when it's not installed so I
> will be thorough:
> 
>         $ cd ~/src/emacs-29.2
>         $ cp lisp/term/xterm.el lisp/term/notworking.el

This is not enough.  You need also to change this line at the end of
notworking.el:

  (provide 'term/xterm)

to say

  (provide 'term/notworking)

Sorry I didn't mention this before.

> While I was setting up this test I'd forgotten that I changed the
> value in each of the three assignments to TN_max_colors in order
> to confirm which condition inside the TERMINFO block was satisfied.
> 
> Before correcting that, running with TERM=notworking reported 8
> colours but with TERM=xterm-direct reported 16 (the extra 8 are all
> black) and said in *Messages*: xterm-register-default-colors:
> Unsupported number of xterm colors (16777214).
> 
> The value ending in 4 pointed to the tigetflag("RGB") block which
> reminded me of the -x option to tic telling it to accept non-standard
> capabilities (of which RGB is one) and I thought perhaps the option
> was also necessary when creating an entry which uses an entry
> which has non-standard capabilities. It turns out that it is and
> that notworking was lacking the extra capabilities.
> 
> So I got excited that maybe I'd found the problem and it was in
> fact bad terminfo data. I did the whole round of tests again but
> this time using "tic -x new.info" instead of plain tic (confirmed
> with infocmp -x that the extended capabilities exist in the new
> terminfo entries notworking and xterm-working).
> 
> The result was identical.

Sorry, I don't think I understand the above.  In particular, I don't
see the value 16777214 anywhere in term.c.  What did I miss?

> Summary:
> 
> My generated terminfo files were wrong. I needed to use tic -x.
> 
> That doesn't matter though because whether or not emacs detects
> "RGB", direct colour availability depends on the $TERM value beginning
> with "xterm-" (and, curiously, that TN_max_colors is valid --- if
> TERM=notworking an invalid TN_max_colors is ignored).

Is that a fact or is it a conclusion from what you observed?  If the
latter, I think we still have stuff to test, see above.

> Creating the file lisp/term/notworking.el had no apparent effect.

See above: the creation was incomplete.

Thanks.