Re: sqlite3

[Alan Mackenzie]

Hello, Karl.

On Tue, Dec 14, 2021 at 14:09:38 -0600, Karl Fogel wrote:
> On 10 Dec 2021, Alan Mackenzie wrote:
> >I've never used org-mode in my life, and only tried out gnus once,
> >around 20 years ago, when it was too slow and too complicated for me.
> >Yet I have to pay the costs of these packages every time I build Emacs
> >bootstrap.

> >I'm not arguing for a removal of these things, which are clearly good.
> >But I think it is reasonable to question the wisdom of adding more
> >inessential stuff.

> You've defined "inessential" in a way that happens to match your 
> particular usage patterns exactly.

Not at all.  I was using the word correctly.  In the words of Fowler,
something is essential when "we have in mind a whole that would not be
what it is to be or is or was if the part in question were wanting; the
essential thing is such that the other thing is inconceivable without
it.".  Emacs without gnus would still be Emacs.  Emacs without Lisp would
not.  In that sense Lisp is essential to Emacs but gnus isn't.

That is an entirely different question as to what is necessary.
Windscreen wipers are necessary on a motor car.  But wheels are
essential.  I'm quite prepared to accept that gnus is necessary for
Emacs.

> But for others, such as myself, Gnus and Org Mode are essential :-).

As above, they're not.  They're necessary.

I question the wisdom of adding more inessential stuff to the Emacs core.
To be perfectly blunt, it is bloat.  I'm not saying that sqlite shouldn't
be added to core.  But I am saying it should be questioned carefully,
particularly by people who are familiar with it (which I'm not).

I feel sqlite has been added to the core, merged into master, with
indecent haste, and without due reflection.  This cannot be good.  I
think there are knowledgeable people here who are against this novelty
and I think their expertise has been disregarded.  That cannot be good
for the Emacs project.

> In a thread from 2020 ([1], "GNU Emacs raison d'etre"), I offered 
> a different understanding of Emacs's essence:

> It is the editing environment that best rewards sustained user 
> investment.

> That differs from your claim that "primarily Emacs is a 
> programmers' text editor".

Clarification here.  I meant an editor used by programmers, not one
programmed by them.

> Programming Emacs is simply the highest level of investment, but it
> doesn't necessarly imply that one is using Emacs *for* computer
> programming -- often, I'm not.  Naturally, users who are programmers
> are going to have the easiest time investing in their Emacs experience,
> both due to skills they have and (probably) due to being
> temperamentally inclined towards patience with such investments. 

> The argument for sqlite3 (or something like it) is that it makes 
> some of those investments easier -- specifically, it makes it 
> easier to do things that involve selective access to large 
> datasets.

Yes.  It comes with a cost, though, and the cost and the benefit have to
be weighed against eachother.

> A concrete example is https://code.librehq.com/kfogel/mailaprop/. 
> Right now, I load all the data in to memory at startup time right, 
> but it's costly -- it noticeably slows down startup.  Fortunately, 
> I don't restart Emacs very often, so this is liveable.  However, 
> if the dataset were 10x or 20x larger, it would be intolerable.

In my mail program, mutt, I load my entire mail collection into memory
every time I start it.  That's around 140,000 mails, it's over 2 GB, and
it loads and threads in 7 or 8 seconds.  I don't know how long it would
take to load into gnus, but I suspect it would be much longer.

> I could of course try out the existing 3rd-party sqlite3 library 
> for Emacs; it's not like there's a huge barrier here.  But still, 
> that would mean adding a dependency that I don't currently have. 
> Whereas if the facility were built into Emacs, the barrier would 
> be lower, so I'd be more likely to experiment with selective 
> loading of such datasets.  Presumably, the same argument applies 
> to others' applications as well, and we'd have the further 
> advantage that everyone would be using a common facility, so it 
> could be improved collaboratively.

Yes, those are the plusses.  The downsides are the bloat (i don't know by
how much), the dilution in Emacs's purpose (it's even less the "do one
thing and do it well" than it was), and I suppose there might even be
negative security implications (it's a database after all).  Also people
will be forced to learn new tools, both inside and outside of Emacs.  I
fear it will create another elite in the Emacs project, those who have
mastered sqlite, similar to, for example, those who can read and write
the cl- library.

> I believe this is one of the points Lars is making.

> Best regards,
> -Karl

> [1] 
> https://lists.gnu.org/archive/html/emacs-devel/2020-05/msg01855.html
>     From: Karl Fogel <kfogel@red-bean.com>
>     To: <emacs-devel@gnu.org>
>     Subject: Re: GNU Emacs raison d'etre
>     Date: Wed, 13 May 2020 11:18:50 -0500
>     Message-ID: <871rnnvmdx.fsf@red-bean.com>

-- 
Alan Mackenzie (Nuremberg, Germany).