Re: persistent data feature

[Matt Armstrong]

Ihor Radchenko <yantar92@gmail.com> writes:

> Stefan Monnier <monnier@iro.umontreal.ca> writes:
>
>>> Another example is Org Roam (https://www.orgroam.com/), which uses
>>> sqlite to manage its index of Org metadata.
>>
>> Could you give details?
>>
>> I don't know what is Org-roam (and the above web-page describes it in
>> terms that don't speak to me very much).  What does it store in the
>> database?  Why did it choose to use sqlite rather than plain text
>> (e.g. simple personal preference on the part of the author, or was
>> a text file tried at first but proved too inefficient, if so which
>> part)?  What kind of database size is common?
>
> Org-roam stores Org headline metadata like headline title, todo-state,
> scheduling info, properties, tags, links to other headings, etc.
>
> AFAIU, they use sqlite for easier integration with third-party
> tools. For example, links between headings can be visualised using
> existing web-libraries (see
> https://github.com/org-roam/org-roam-ui/blob/main/package.json)
>
> As for the DB size, I am not sure. However, it should scale with the
> number of headings. You may think of 10-100k headline DB records and a
> multiple of that number for links between headings.

The author gave a presentation on org-roam in EmacsConf 2020:
https://emacsconf.org/2020/talks/16/.  It is a Zettelkasten note taking
system for Emacs built on top of Org.

I believe that org-roam uses sqlite primarily for performance, as an
index over many the small org files the packages encourages users to
create.  I suspect that the use case is much like Gnus.

I have a memory (that I can't verify as fact now) of the early version
using awk to extract the needed info from the org files directly, but
this proved to be slow in practice.

I suspect that people have found ways to use the database for other
things is a secondary benefit.