bug#59346: Adding sqlite-backup

[Andrew Hyatt]

On Fri, Nov 18, 2022 at 9:17 AM Eli Zaretskii <eliz@gnu.org> wrote:

> From: Andrew Hyatt <ahyatt@gmail.com>
> Date: Fri, 18 Nov 2022 09:05:58 -0300
> Cc: 59346@debbugs.gnu.org
>
>  Thanks, but I'm not sure Emacs should support DB administration
>  functions of sqlite.  Aren't there utilities out there which can be
>  used for this?  Why should we have this built-in in Emacs? why not
>  simply use shell-command or somesuch to invoke the necessary external
>  program?
>
> Good questions. Let me respond to your two questions: why does emacs need to be involved in backups,
> and why can't it control backups some other way?
>
> Emacs already has a file backup facility, but if emacs is primarily working with data in sqlite, there is no
> mechanism for backups, which seems scary to me.  I'd like modules that rely on sqlite for data to be able to
> back up their data, because the user would like to have some security, knowing if something goes wrong
> they can always restore a recent backup.

A database is not like a normal file, and so the fact that Emacs has
backups doesn't seem to be a reason good enough to extend the backups
to DB operations.  We don't bother with this when we send email or do
other operations.  We also have auto-save and file-locks for normal
file, but not for DBs.  And a DB is not a file, it is a (large)
collection of tables and records, and Emacs deals with at most a
single table at a time, AFAIU. 

More generally, DB administration is outside of the Emacs scope, and
how to do that properly is outside our expertise.  There's more to it
than just backing up the DB.

So I don't think we should extend the Emacs sqlite3 support in this
direction.

I disagree, but I understand this is just a difference of opinion, so I'm happy to let this patch sit here until there's more demand for this sort of thing. 

But if you are interested in furthering the conversation, which has a lot to do with what I'm talking about in my upcoming (and already recorded) Emacs conference talk, I don't think the distinction you are making between files and databases is important.  Let's say I wanted to store, use and otherwise manipulate contact information with emacs.  Right now that sort of thing happens with a file with emacs-readable lists inside, but it's really more natural to do it with a database.  The fact that it's done via a file now seems historical, especially now that we have sqlite.  I expect that sqlite will be used for this sort of thing more often.  However, if there's no database option, then maybe not, since it won't be seen as reliable as the filesystem.  I don't think it is reasonable to expect users to figure out how to do this, even if the info is easy to find.  I'd like us to be able to encourage the use of sqlite for data, and give the user a good experience when sqlite is the backend.

 

> But why can't we do this by invoking backup via the binary?  Certainly a possibility, and in fact that's how I
> would implement it if the sqlite library was instead emacsql-sqlite.  However, I have no idea how to locate the
> binary; that isn't part of the current built-in sqlite implementation.  For good reason, too, it's a built-in
> implementation, including a variable holding the executable path seems odd.  I could add one, but this
> solution seems better to me, since it doubles down on the built-in sqlite, and doesn't provide an alternative
> mechanism for doing things with sqlite.

This SO article:

  https://stackoverflow.com/questions/25675314/how-to-backup-sqlite-database

(which is the first hit I get if I "how to backup sqlite database"
into the browser search box) says that you can backup the DB by using
the sqlite3 executable.  I think it is reasonable to expect users who
want to backup their DB to have this executable and use it for that
purpose (and other purposes, as they need).  I do have this executable
here, FWIW.

So I think these needs should be fulfilled "by other means", not by
Emacs Lisp programs.