Re: [PATCH] Add a source for beets library databases

[Ian Eure]

Yoni Rabkin <yoni@rabkins.net> writes:

> Daniel Semyonov <daniel@dsemy.com> writes:
>
> > Attached is a patch which adds a source which reads a beets 
> > library
> > database using Emacs' built-in SQLite support.
>
> Does beets provide a stable api to access its SQL database 
> directly?

Depends on your definition of stable API.  The CLI tooling is (as 
far as I know) stable.  I believe the dbcore package of the Beets 
source is the only official implementation of the lower-level 
database access mechanism.  Since beets is written in Python, I 
don’t think that’s helpful for EMMS.


> Can a future (or old) version of beets on someone's machine 
> break the Emms code by having tables in different place or 
> different keys?

I think the schema is fairly stable at this point, but both 
situations are possible.  It doesn’t appear to have any explicit 
versioning, but the classes representing things stored in the DB 
will update the schema when appropriate -- look for _make_table 
functions in dbcore/db.py.


> If beets is built to expect other programs to access its 
> database directly then this would work.

Beets uses SQLite, which doesn’t support concurrent access at all, 
even read-only.  In practice, this isn’t much of a problem, 
because beets is run infrequently, like when you want to add new 
music.

There are workarounds, such as copying the database or hard 
linking it, both of which defeat SQLite’s locking mechanism. 
Though hard linking isn’t portable to non-unix-like platforms, and 
is fairly dangerous -- defeating the locking means that if either 
EMMS or Beets writes to the .sqlite3 file, the other accessor will 
likely blow up.


> Otherwise, I'm unsure of how the beets code may react to 
> something
> altering the contents of its SQL tables behind its back. You 
> also don't
> want Emms to break beets by editing its tables in a way that 
> version of
> beets isn't expecting.
>
> Does it make more sense to either:
>
>    * Use the "official" beets interface to access the contents 
>    of the
>      database instead.

Since Beets is written in Python, I’m not sure how this would work 
-- are you thinking that EMMS would execute `beets ls' type 
commands and format the output in a way it can consume?  Due to 
the wide variety of weird band / song titles, this seems like an 
approach that’s likely to be unreliable for music with names in 
any non-Latin character set.

If beets had a way to format results as JSON, that might be a 
middle ground.  It doesn’t appear to, but upstream may be open to 
the idea -- though it would set a (very recent) lower bound on the 
supported Beets version.


>    * Use the SQLite support in Emacs to import/update an Emms 
>    SQL
>      database from the beets one. This mirroring will mean that 
>      the
>      beets database can change, or even go away, and the Emms 
>      database
>      will remain.
>
>      This second option is effectively equivallent to Emms 
>      implementing
>      its cache in SQLite.
>
> Can you speak to these issues?

Personally, I would recommend the second approach, as it seems 
straightforward to implement & wouldn’t require invasive changes 
to EMMS to support.  And I’d think that accessing a SQLite 
database like that would be significantly faster than scanning all 
the files in the library.

Side note, I’ve come to appreciate how EIEIO lets Emacs Lisp 
programs easily add/change implementations of an interface, and 
this seems like a good match for some of EMMS’s needs.  It would 
be a biggish / BC breaking change, but have you considered porting 
the player backend stuff to a system like that?

 — Ian