[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [O] How to make agenda generation faster
From: |
Ihor Radchenko |
Subject: |
Re: [O] How to make agenda generation faster |
Date: |
Wed, 17 Oct 2018 15:04:58 +0800 |
> I've thought about this for a while. It seems to me that the issue is
> that Org buffers are, of course, plain-text buffers. There is no
> persistent, in-memory representation other than the buffer, so whenever
> Org needs structured/semantic data, it must parse it out of the buffer,
> which is necessarily rather slow. If there were a way to keep an
> outline tree in memory, parallel to the buffer itself, that would allow
> operations like search, agenda, etc. to be greatly sped up.
FYI
A while ago I saw some cache implementation in org-element.el.
Take a look at org-element--cache variable definition and the code
below.
````
(defvar org-element--cache nil
"AVL tree used to cache elements.
Each node of the tree contains an element. Comparison is done
with `org-element--cache-compare'. This cache is used in
`org-element-at-point'.")
````
Best,
Ihor
Adam Porter <address@hidden> writes:
> Nicolas Goaziou <address@hidden> writes:
>
>>> my understanding is that code that runs with lexical-binding enabled
>>> is generally faster.
>>
>> Not really. But it's certainly easier to understand since it removes one
>> class of problems.
>
> From what I've read, the byte-compiler can optimize better when
> lexical-binding is used.
>
>> Instead of re-inventing the wheel, or putting efforts into a
>> wheel-like invention, wouldn't it make sense to actually work on Org
>> Agenda itself?
>>
>> So again, wouldn't it be nice to think about Org Agenda-ng?
>
> As a matter of fact, what's now called org-ql-agenda was originally
> called org-agenda-ng. I factored org-ql out of it and realized that it
> should probably be its own, standalone package. Then I renamed
> org-agenda-ng to org-ql-agenda, so I could reasonably keep them in the
> same repo, and because I don't know if I will ever develop it far enough
> to be worthy of the name org-agenda-ng. It started as an experiment to
> build a foundation for a new, modular agenda implementation, and maybe
> it could be.
>
>> I didn't look closely at org-ql, but I had the idea of splitting the
>> Agenda in two distinct parts. One would be responsible for collecting,
>> possibly asynchronously, and caching data from Org documents. The other
>> one would provide a DSL to query and display the results extracted from
>> the output of the first part. The second part could even be made generic
>> enough to be extracted from Org and become some part of Emacs.
>> Displaying filtered data, maybe in a timeline, could be useful for other
>> packages. Unfortunately, I don't have time to work on this. Ah well.
>
> I've thought about this for a while. It seems to me that the issue is
> that Org buffers are, of course, plain-text buffers. There is no
> persistent, in-memory representation other than the buffer, so whenever
> Org needs structured/semantic data, it must parse it out of the buffer,
> which is necessarily rather slow. If there were a way to keep an
> outline tree in memory, parallel to the buffer itself, that would allow
> operations like search, agenda, etc. to be greatly sped up.
>
> But how would that work in Emacs? Theoretically, we could write some
> code, applied on self-insert-command, to update the "parallel tree
> structure" as the user manipulates the plain-text in the buffer
> (e.g. add a new node when the user types a "*" to create a new heading),
> and also apply it to functions that manipulate the outline structurally
> in the buffer. But, of course, that sounds very complicated. I would
> not relish the idea of debugging code to keep a cached tree in sync with
> a plain-text buffer outline. :)
>
> Besides that, AFAIK there would be no way to do it asynchronously other
> than calling out to a child Emacs process (because elisp is still
> single-threaded), printing and reading the data back and forth (which
> would tie up the parent process when reading). Maybe in the future
> elisp will be multithreaded...
>
> Anyway, org-ql tries to do some of what you mentioned. It does
> rudimentary, per-buffer, per-query caching (as long as the buffer is not
> modified, the cache remains valid), which helps when there are several
> Org files open that are referred to often but not as often modified.
> And the query and presentation code are separated (org-ql and
> org-ql-agenda).
>
> I don't know how widely it's used, but the repo is getting some regular
> traffic, and I'm using it as the backend for my org-sidebar package.
> I'd be happy if it could be made more generally useful, or if it could
> be helpful to Org itself in some way. Contributions are welcome.
>
>
signature.asc
Description: PGP signature
- Re: [O] How to make agenda generation faster, (continued)
- Re: [O] How to make agenda generation faster, Marcin Borkowski, 2018/10/10
- Re: [O] How to make agenda generation faster, Samuel Wales, 2018/10/10
- Re: [O] How to make agenda generation faster, Michael Welle, 2018/10/11
- Re: [O] How to make agenda generation faster, Marcin Borkowski, 2018/10/11
- Re: [O] How to make agenda generation faster, Samuel Wales, 2018/10/11
- Re: [O] How to make agenda generation faster, Marcin Borkowski, 2018/10/14
Re: [O] How to make agenda generation faster, Adam Porter, 2018/10/09
- Re: [O] How to make agenda generation faster, Nicolas Goaziou, 2018/10/09
- Re: [O] How to make agenda generation faster, Marcin Borkowski, 2018/10/10
- Re: [O] How to make agenda generation faster, Adam Porter, 2018/10/16
- Re: [O] How to make agenda generation faster,
Ihor Radchenko <=
- Re: [O] How to make agenda generation faster, Nicolas Goaziou, 2018/10/17
- Re: [O] How to make agenda generation faster, Adam Porter, 2018/10/17
- Re: [O] How to make agenda generation faster, Nicolas Goaziou, 2018/10/18
- Re: [O] How to make agenda generation faster, stardiviner, 2018/10/18
- Re: [O] How to make agenda generation faster, Adam Porter, 2018/10/19
- Re: [O] How to make agenda generation faster, Nicolas Goaziou, 2018/10/20
Re: [O] How to make agenda generation faster, Marcin Borkowski, 2018/10/10
Re: [O] How to make agenda generation faster, Julius Dittmar, 2018/10/09