bug#35383: 27.0.50; Complete process of decoding Gnus group names

[Eric Abrahamsen]

Katsumi Yamaoka <yamaoka@jpl.org> writes:

> On Thu, 25 Apr 2019 09:10:13 -0700, Eric Abrahamsen wrote:
>> On 04/25/19 08:48 AM, Katsumi Yamaoka wrote:
>
>>> Warning: Warning - invalid active:
>
>>> for the nnnil method, that is my `gnus-select-method'.  Here are
>>> the contents of ~/News/agent/nnnil/agent.lib/active:
>
>>> --8<---------------cut here---------------start------------->8---
>>> ;; -*- encoding: utf-8-emacs; -*-
>
>>> --8<---------------cut here---------------end--------------->8---
>
>>> Why the warning is issued is to run (read (current-buffer)) at
>>> the beginning of the contents.  This is actually an error but
>>> `condition-case' conceals it.
>
>> Hmm, this is all done in a temp buffer,
>
> Yes.  When launching Gnus, the whole contents of the active file
> are read into the " *nntpd*" buffer, copied into the temp buffer,
> and parsed (see the flow summary attatched in the bottom of this
> message for how Gnus behaves when launching).
>
>> with
>> `insert-buffer-substring' -- I wonder if the encoding cookie will even
>> be honored in this case?
>
> No, it's useless of course.  Moreover, --- I changed my idea
> (patching the `gnus-active-to-gnus-format' function so as to
> ignore the coding cookie) --- I come to think that the active
> file should not contain the ones other than the active infos.
> Gnus indeed ignores the coding cookie when parsing active, but
> it is due to just a lucky side effect of `read':
>
> (read ";; coding cookie\n\nactive_info\n") => active_info
>
> I.e., `read' ignores comments in the ELisp style and whitespace.
> However, in the first place, the active file is neither an ELisp
> file nor there is no agreement for a comment style in it.  So, I
> think it is better to bind `coding-system-for-(read|write)' while
> reading and writing the active file rather than adding a coding
> cookie.  Though binding `coding-system-for-(read|write)' would
> probably be unnecessary since `gnus-write-active-file' binds
> `coding-system-for-write' to `nnmail-active-file-coding-system',
> and `gnus-agent-save-active' binds `coding-system-for-read' to
> `gnus-agent-file-coding-system' that defaults to `utf-8-emacs'.
> Therefore, adding a coding cookie was originally unnecessary,
> wasn't it?
>
> Here are how Gnus reads the active file for the nnnil method
> observed in my system.  Note that `gnus-agent' is t (the default).
>
> (gnus 1)
>   [...]
>   (gnus-setup-news nil t nil)
>     (gnus-get-unread-articles 1 nil)
>       (require 'gnus-agent)
>       (with-current-buffer " *nntpd*"
>         (gnus-read-active-file-1 '(nnnil) nil)
>           (gnus-active-to-gnus-format '(nnnil) hashtb nil t)
>             (gnus-agent-save-active '(nnil))
>               (gnus-agent-write-active "active-file" hashtb)
>                 ;; Add a coding cookie.
>                 (gnus-write-active-file "active-file" hashtb nil)
>               (erase-buffer)
>               (nnheader-insert-file-contents "active-file")
>             (_copy to_ " *nntpd*")
>             (_parse it_)

Hang on, let me slow down here.

The goal is to have Gnus default to writing its active files in
'utf-8-emacs, unless the user has specifically requested otherwise.
`nnmail-active-file-coding-system' governs the "mail" type servers, and
`gnus-agent-file-coding-system' governs the agent. Currently those two
options default to 'raw-text, we'd like them to default to 'utf-8-emacs.

But the active files have to be read correctly, as well, even by a user
who is loading this new code for the first time. That was my original
idea for the coding cookie, though obviously I was confused. Both those
options are used as the value of `coding-system-for-read' when reading
active files. So far I don't have a good solution for "upgrading" old
active files to 'utf-8.

I also realized that there's a `nnmbox-active-file-coding-system' that
defaults to 'binary.

What I find a little perplexing is that, testing with emacs 26, an nnml
active file with multibyte group names gets written to a file that my
terminal recognizes as utf-8, despite the fact that `nnmail-save-active'
sets 'coding-system-for-write' to 'raw-text, and disables multibyte. But
maybe, as Eli mentioned, that has to do with my locale...?

> P.S.
> I'll be not so active in the net for about ten days because of
> the national holidays assoc with the era name changing in Japan.

Hope it's a happy time in Japan! I'm in no rush over here.

Eric