bug#69517: [PATCH] Make gnus cache work with group names having '/'

[Eric Abrahamsen]

Eric Abrahamsen <eric@ericabrahamsen.net> writes:

> James Thomas <jimjoe@gmx.net> writes:
>
>> Eli Zaretskii wrote:
>>
>>>> Date: Sun, 03 Mar 2024 07:22:29 +0530
>>>> From:  James Thomas via "Bug reports for GNU Emacs,
>>>>  the Swiss army knife of text editors" <bug-gnu-emacs@gnu.org>
>>>>
>>>> Tags: patch
>>>>
>>>> Reproduction steps:
>>>>
>>>> - Setup Gnus with any group name having a slash ('/') such as
>>>>   "[Gmail]/Drafts" or an Atom feed (they usually have slashes) using the
>>>>   patch in bug#66188.
>>>>
>>>> - Press '*' on a message in the group.
>>>>
>>>> - Do (info "(gnus) Creating a Virtual Server")
>>>>
>>>> - Open the above from the Server buffer; RET on the new group fails.
>>>>
>>>> A patch is attached. I couldn't find the problematic commit or its
>>>> original branch (where it was a consolidated merge from) but
>>>> 'gnus-use-long-file-names' is apparently not meant for backends: it
>>>> can't even be customized with that 'not-cache' option. I think this is
>>>> the right way to solve it: the other lines removed in this patch are
>>>> even older, but they were never being called due to the above reason.
>>>
>>> Eric, could you please review the patch and install if it's okay?
>>>
>>> Thanks.
>>
>> There's a small caveat after applying this:
>>
>> Before this patch, cache entries would've effectively ignored
>> 'nnmail-use-long-file-names' (nil by default) and used long names. After
>> this patch new entries would honor it, resulting in an extra directory
>> tree for the same group. But only the original one would be opened (due
>> to [[./lisp/gnus/nnmail.el::;; If this direc]]). To fix it, one would
>> have to copy all the files in from the original directory into the new
>> one (and retain the active file entry). Or of course, change the above
>> variable (but that could have other implications depending upon one's
>> configuration).
>
> Thanks for reporting this! Obviously something is wrong here, but I get
> different results from your reproduction recipe (I'm running master):
> when I create the cache nnml server and hit RET on it, I get:
>
> K      1: nnml:left.right
> K      1: nnml:left/right
>
> I am able to subscribe to both groups. When I return to the *Group*
> buffer, I can enter the version of the group with the ".", and read the
> cached article, while attempting to enter the version with the slash
> gives me "Invalid group (no such directory)".
>
> On disk, the "News/cache/active" file starts out with the version with
> the ".", and after I create the extra cache server it contains both, it
> looks like:
>
> "nnml:left.right" 1 1 y
> "nnml:left/right" 1 1 y
>
> Meanwhile, the directory structure "News/cache/nnml:left/right/1" is
> created.
>
> The "." in the active file comes from `gnus-cache-generate-active`, and
> that mechanism seems to be working fine, at least in my case -- do you
> not see that version of the group?
>
> To be honest I have no idea what's happening in `gnus-cache-file-name'.
> I'd be hesitant to remove all that code without understanding it better
> And I don't think we can argue that that code is "dead" just because it
> is behind an option value that isn't one of the customization options.
>
> Anyway, it would be good to first know why you're not seeing the "."
> version of the offending filename, and then I guess try to work out
> what's going on with the file names. It kind of looks like a collision
> between two different mechanisms -- one that translates "/" to "_" to
> prevent unwanted filesystem hierarchies, and one that translates between
> "/" and "." in order to convert newsgroup-name dot hierarchies into
> filesystem hierarchies. In this case, it seems like "long names" and
> newsgroup hierarchies shouldn't be applicable at all.

I was briefly optimistic about the possibility that your previous patch,
on bug#65467, might help address this issue. That patch did make it
possible for nnml groups with slashes in the name to work correctly
(they didn't, previously), but it still doesn't address this particular
scenario.

Meanwhile, I realize that my hesitation on that bug report is bogus, as
everywhere else we already have active files with quotes in them, and
`gnus-active-to-gnus-format' does a fair amount of work to handle
various quoting situations.

The real problem (well, one of the real problems) is that we should just
have two central routines for reading and writing active files, so that
there are only two places to keep in sync. Instead we have those two
places, and then a smattering of other functions in other places that do
something similar, and also have to be kept in sync, and I haven't done
that. At the very least I'll need to apply your patch from 65467.

The other real problem is that gnus-cache is confused about whether it
wants to be a select method, or a modified version of article saving.
The presence of `gnus-use-long-file-name' indicates the latter, but the
manual's instructions about the nnml select method indicates the former.

I think it should be a select method, which means that the group
directory should be created in "News/cache" the same way it is created
at the top level: with the "/" replaced by "_", and everything else
using the proper "left/right" group name. Perhaps "long file names" can
still play a role, but if so that should be via
`nnmail-use-long-file-names', and gnus-cache in general should behave
like a nnmail backend.

Eric