Re: [patch suggestion] Mitigating the poor Emacs performance on huge org files: Do not use overlays for PROPERTY and LOGBOOK drawers

[Nicolas Goaziou]

Hello,

Ihor Radchenko <address@hidden> writes:

> Apparently my previous email was again refused by your mail server (I
> tried to add patch as attachment this time).

Ah. This is annoying, for you and for me.

> The patch is in
> https://gist.github.com/yantar92/6447754415457927293acda43a7fcaef

Thank you.

>> I have finished a seemingly stable implementation of handling changes
>> inside drawer and block elements. For now, I did not bother with
>> 'modification-hooks and 'insert-in-font/behind-hooks, but simply used
>> before/after-change-functions.
>>
>> The basic idea is saving parsed org-elements before the modification
>> (with :begin and :end replaced by markers) and comparing them with the 
>> versions of the same elements after the modification.
>> Any valid org element can be examined in such way by an arbitrary
>> function (see org-track-modification-elements) [1].

As you noticed, using Org Element is a no-go, unfortunately. Parsing an
element is a O(N) operation by the number of elements before it in
a section. In particular, it is not bounded, and not mitigated by
a cache. For large documents, it is going to be unbearably slow, too.

I don't think the solution is to use combine-after-change-calls either,
because even a single call to `org-element-at-point' can be noticeable
in a very large section. Such low-level code should avoid using the
Element library altogether, except for the initial folding part, which
is interactive.

If you use modification-hooks and al., you don't need to parse anything,
because you can store information as text properties. Therefore, once
the modification happens, you already know where you are (or, at least
where you were before the change).

The ideas I suggested about sensitive parts of elements are worth
exploring, IMO. Do you have any issue with them?

>> For (2), I have introduced org--property-drawer-modified-re to override
>> org-property-drawer-re in relevant *-change-function. This seems to work
>> for property drawers. However, I am not sure if similar problem may
>> happen in some border cases with ordinary drawers or blocks. 

I already specified what parts were "sensitive" in a previous message.

>> 2. I have noticed that results of org-element-at-point and
>> org-element-parse-buffer are not always consistent.

`org-element-at-point' is local, `org-element-parse-buffer' is global.
They are not equivalent, but is it an issue?


Regards,

-- 
Nicolas Goaziou