bug#36714: 27.0.50; Gnus nnmaildir taking long time to recursively visit sub-Maildirs

[Eric Abrahamsen]

On 07/18/19 22:48 PM, Jean Louis wrote:
> * Eric Abrahamsen <eric@ericabrahamsen.net> [2019-07-18 22:04]:
>> I'm assuming your value of `gnus-check-new-newsgroups' is at its default
>> of 'ask-server. Try setting it to nil. That will (should) at least
>> prevent Gnus from scanning all the folders at startup. It doesn't solve
>> the underlying problem, but in your case it
>> might avoid it.
>
> Thank you.
>
> I did try to set it to nil, it is set. However,
> "Checking new news" is seen and Gnus is working
> with hard disk.
>
> So it does not prevent it to scan Maildirs.

I guess that's not too surprising. If Gnus has never actually been able
to get off the ground, it's probably doing an initial scan of some sort.

> gnus-select-method is a variable defined in ‘gnus.el’.
> Its value is (nnmaildir "" (directory "~/Maildir"))
>
>> The final setup you want is to be subscribed to the main folder, but
>> unsubscribed from the sub maildirs. The question is, can you get to the
>> point where you can do that without first having Gnus spend hours
>> scanning a million directories. Try the above fix and see if Gnus will
>> show you all the other (unwanted) directories.
>
> It does not work.
>
>> We're talking about two different things here. One is defining each of
>> the maildirs as a separate server. So:
>>
>>  '(gnus-secondary-select-methods
>>     '(
>>     (nnmaildir email1
>>      (directory /home/data1/protected/Maildir/email1@example.com/))
>>     (nnmaildir email2
>>      (directory /home/data1/protected/Maildir/email2@example.com/))
>>     (nnmaildir email3
>>      (directory /home/data1/protected/Maildir/email3@example.com/))))
>>
>> Etc.
>>
>> The other thing you want -- quick switching to a particular group -- can
>> certainly be done, but will require a little bit of elisp. Something
>> like (totally untested):
>>
>> (let* (( email "ss@rcdrrun.com")
>>        (account (car-safe (split-string email @))))
>>   (when account
>>     (gnus-group-read-group
>>      nil t (concat nnmaildir+ account :INBOX))))
>
> I will try that, but I cannot get it without
> starting to run or index whatever.
>
> I was thinking that gnus-secondary-select-methods
> has to be set before I start reading the folder,
> and that above looks that you think that way.

Yes, you should try setting your secondary select methods to
one-per-account before you try any of the rest of this.

> I have tried this
>
> (let* ((email "person@example.com")
>       (gnus-secondary-select-methods '((nnmaildir email (directory (concat 
> "~/Maildir/" email))))))
>    (gnus-group-read-group nil t (concat "nnmaildir+" email :INBOX)))
>
>
> but I am getting this below and I used real email
> address. Do I need to assign the group somehow?

First of all, this isn't going to work until you've got Gnus into a
basic functioning state -- ie, it's already done a successful scan of
your nnmaildir backends. I would leave this part aside until you've got
Gnus working normally. (Also you can't reference a variable inside a
quoted form without using backquoting and unquoting, and let-binding
`gnus-secondary-select-methods' is asking for trouble, and :INBOX needs
to be a string, but really don't bother messing with this at all until
Gnus is functioning.)

Second of all... unfortunately I don't know this code well enough to
figure it out without sitting down with a chunk of time. So far as I can
tell, nnmaildir scans the maildir directories using `directory-files',
which by itself is very quick. Presumably what's slow is registering all
the messages inside the directories, and I don't immediately see a way
of preventing that.

A dirty hack you might try is: temporarily remove the other directories
from one of the maildir accounts, let Gnus index that, and then (while
making sure that `gnus-check-new-newsgroups' is nil) move the other
directories back in. It's possible that Gnus will ignore them, but I'm
really not sure -- I would check with just one of your four accounts
first.

Until I or someone else has time to dig deeper, that's the best I can
suggest, sorry...

Eric