[Orgmode] Re: Having (too) many files in org-agenda-files

[Sébastien Vauban]

Hi Carsten,

Carsten Dominik wrote:
> On Sep 28, 2010, at 8:45 PM, Sébastien Vauban wrote:
>> Of course, I have many, many files in Org mode. All files I write (or touch)
>> in fact.
>>
>> Of course, I would like to search through my files at some point in time. I
>> even would like to search through your files at some point in time, I mean
>> through =org-mode/contrib/babel= and =Worg= for example. Hence, I put them
>> in =org-agenda-files=.
>>
>> The problem is the load-time of my Emacs, now 221 seconds, coming from 20
>> seconds before the heavy use of Org...
>>
>> You'll tell me: not a problem, you do that only once a day, and you use
>> Emacs client/server for the rest of the time. True. A bit, because I
>> sometimes have to restart Emacs for testing a fresh one (not impacted by
>> defvars already defined, or deffaces, etc.).
>>
>> Having to wait almost 4 minutes is a real pain. So, here my
>> comments/questions:
>>
>> - Isn't it possible to delay the fontification/ispell/etc. to when we
>>   really display (i.e., pop up) the buffer? I guess this must be a major
>>   component of the time this takes.
>>
>> - Couldn't we have 2 vars: =org-agenda-files= for the files you know you
>>   want have scanned for the agenda construction, and an extra list such as
>>   =org-search-files= for files not containing any dates? Then, some time
>>   would have to be taken when =C-c a s=, but not before. And not if you
>>   don't search for anything in your Org files during that Emacs session...
>
> 4 minutes of startup time is entirely unacceptable.  And I think
> you need to identify what is causing this.
>
> First of all, unless you are having to totally old Version of Emacs,

Nope. GNU Emacs 23.1.1 (i486-pc-linux-gnu, GTK+ Version 2.20.0)
of 2010-03-29 on rothera, modified by Debian


> fontification at display time is standard, I believe.

It seems not, from what I see in the Messages buffer. I really don't have the
impression of having fiddled with that, really.


> You must be doing something strange, like forcing Emacs to load
> all those files in org-agenda-files for example,

That, yes!  I should have explicitly written it, but -- yes -- I do have:

--8<---------------cut here---------------start------------->8---
(org-agenda-list)
--8<---------------cut here---------------end--------------->8---

in my .emacs.


> and maybe forcing global fontification for each file or so.

To repeat myself, no to that question. But ispell and flyspell are called for
Org files. Maybe there are interactions?


> About a separate variable for search file, that does of cause exist:
> org-agenda-text-search-extra-files

Of course!

Doing this:

--8<---------------cut here---------------start------------->8---
(setq org-agenda-files
      (append (directory-files org-directory t "^[^\\.].*\\.org$")
              (if (file-exists-p "~/Projects/")
                  (directory-files "~/Projects/" t "^[^\\.].*\\.org$")
                nil)))

(setq org-agenda-text-search-extra-files
      (append
              (if (file-exists-p "~/Public/")
                  (directory-files "~/Public/" t "^[^\\.].*\\.txt$")
                nil)
              (if (file-exists-p "~/Public/www.mygooglest.com/source/sva/")
                  (directory-files "~/Public/www.mygooglest.com/source/sva/" t 
"^[^\\.].*\\.txt$")
                nil)
              (if (file-exists-p "~/Examples/Org-scraps/")
                  (directory-files "~/Examples/Org-scraps/" t 
"^[^\\.].*\\.txt$")
                nil)
              (if (file-exists-p 
"~/Downloads/emacs/site-lisp/org-mode/contrib/babel/")
                  (directory-files 
"~/Downloads/emacs/site-lisp/org-mode/contrib/babel/" t "^[^\\.].*\\.org$")
                nil)
              (if (file-exists-p "~/Downloads/emacs/site-lisp/Worg/")
                  (directory-files "~/Downloads/emacs/site-lisp/Worg/" t 
"^[^\\.].*\\.org$")
                nil)
              ))
--8<---------------cut here---------------end--------------->8---

reduces my load time from 221 seconds down to 92 seconds. Already a huge diff!

Thanks...

Best regards,
  Seb

-- 
Sébastien Vauban