Re: sqlite3

[Qiantan Hong]

>> My only remark is that sqlite is usually used for exactly same purpose as 
>> what
>> Emacs already has: customize. Your example:
>> 
>>> (setf (persistent-data :namespace "emoji" :key "favorites") 
>>> emoji--favorites)
>> 
>> looks to me like something one would use customize for.
> 
> Is this historic use of customize not just a result of the lack
> something like sqlite3?
Or it could be the other way around. The prominent use of sqlite3
is that other environments (especially the more “static” ones) lack
customize, or the ability to configure the environment within itself
in general, thus relying on a central external store.

I really don’t like the idea of introducing an external UNIX-y 
dependency that significantly overlaps with Emacs Lisp itself,
it’s almost like saying “let’s make Emacs programmable using
JavaScript, because JavaScript is faster, safer, etc etc”.

We don’t need the sql language. In most cases it’s no better than 
MAPCAR-ing and FILTER-ing, and it’s strictly no better than just 
embedding a Prolog interpreter in Emacs Lisp.

What we really need is a faster persistence mechanism.
Anything that does read/write incrementally will be
lightning fast comparing to the current practice of printing/reading
plain s-exp, and is more than enough. 
Gdbm would fit perfectly if it weren’t, allegedly, too unstable.
Alternatively, I think it would be very doable to just implement 
a persistent hash-table with plain Emacs Lisp.
If I’ve got some free time I’ll probably do it.

> I think the more significant benefit is to be able to correctly handle
> cases where several Emacs sessions modify the persistent data.
> I'm thinking of things like `savehist`.
I think the problem of sharing mutable data between Emacsen
is orthogonal to persistence. It could be handled very well
within the images. crdt.el did a good job in sharing mutable
buffer (together with text properties, overlays, marks etc),
and I could add support for sharing key value maps at some point.
We don’t need to outsource it to some C program.

> Also in general since we can read/write lisp objects easily in elisp it is not
> hard to serialize/deserialize a hashmap or an assoc list as an application
> key-value db. I don't know how many application does this, but I have used it
> myself, and it felt trivial to use in my app. Objects that are not 
> serializable
> will not be serializable to sql either, without some extra work, so there is 
> not
> much win to use sqllite db instead of hashmap. 
That’s half of my point — we don’t need much (like a whole SQL implementation)
more than what we already have. I suspect simply dump out the whole
hash table to say gdbm won’t do a lot better than printing s-exp directly 
though.
The most important optimization is to write it incrementally.

Best,
Qiantan