Re: frescobaldi extensions [was: python-ly, ly.indent]

[Urs Liska]

Hi Mason,

18. August 2019 01:05, address@hidden schrieb:

> On 08/17, Urs Liska wrote:
> 
>> ...
> 
> Thanks. While I personally prefer command line tools, I would consider
> turning this into a Frescobaldi extension if that would mean the
> difference between something only being useful to me versus potentially
> being useful to others. 

OK. From my experience I'd say it is a worthwile but non-trivial effort to make 
things generally useful and available. You always have to strike a balance 
between enabling *arbitrary* use cases and imposing complex configuration 
"costs" on the user.

> Some questions:
> 
> I am planning to include several openLilyLib packages as submodules.
> Edition Engraver integration is a particularly high priority, because
> one goal is to be able to maintain editions of scores and parts for
> different paper and tablet sizes. I could see a dependency on
> openLilyLib packages as being a problem for a Frescobaldi extension.

That's correct and actually an example of the undue burdens I meant in the 
comment above.

> Unless the extension provides the relevant packages, the user will need
> to supply the correct version of those packages, which will be hard
> unless openLilyLib starts providing versioned releases of packages. (It
> looks like ScholarLY already does,[1] but Edition Engraver does not.)
> Are there plans to integrate any OLL packages into Frescobaldi (or
> Lilypond proper, for that matter)?

Well, that's basically two questions, one directed at openLilyLib, the other at 
Frescobaldi.

With openLilyLib I would long have wanted to invest more work (or have 
contributors) to move it in a more structured direction, along with a proper 
strategy for consistent documentation, a website and also a clearer outside 
interface with releases and such.

One thing that has been done in that direction is requiring an openLilyLib 
package to have a metadata file package.cnf with details such as name and 
description, but also listing the included submodules (scholarLY for example 
has modules like annotate, choice or editorial-markup), package version, API 
version and dependencies. 
https://github.com/openlilylib/scholarly/blob/master/package.cnf is scholarLY's 
metadata file.
Andrew Bernard has written a parser for a simple configuration language used 
for this 
(https://github.com/openlilylib/oll-core/blob/master/scheme/oll-core/internal/vbcl.scm),
 which I've ported to Python for use in Frescobaldi 
(https://github.com/frescobaldi/frescobaldi/blob/master/frescobaldi_app/vbcl/__init__.py).

The trigger for that development actually was (and that gets to the second 
issue) my intention to integrate openLilyLib support into Frescobaldi. I had 
already started with it but had to stop at some point. From the attached 
screenshot you can see that it would list all packages found in a root 
directory and display their metadata, while the management functions accessible 
through the buttons to the right have not been implemented yet.

Actually, this feature is a prime use case for Frescobaldi extensions: while I 
want that functionality seamlessly integrated in the Frescobaldi environment 
it's not general enough to be added to Frescobaldi itself. The moment to add 
openLilyLib support to Frescobaldi would be when openLilyLib would natively be 
added to LilyPond, which is not the goal.

So if you write an extension that deals with openLilyLib the extension could 
simply name the not-yet-written openLilyLib extension as a dependency. The 
openLilyLib extension would then have to provide an interface (something I 
realized just now) to manage extensions programmatically (i.e. your extension 
can for example require the edition-engraver package).

###

With regard to "project management" extensions I would like to write one (or 
two) "base extensions" that provide reusable functionality for two common use 
cases. As far as I can tell there are two fundamental approaches to large 
repertoires (e.g. your whole collection of scores, a work with numerous 
sub-scores or the like): there is either a given repertoire that has to be 
completed or there is a loose collection of scores. In the first case there has 
to be a list of scores that the interface can navigate and handle (this is what 
one of the screenshots in my previous post shows), while in the second the 
repertoire is inferred from a given directory tree (and you can simply add new 
works by adding subdirectories).

So far I have implemented one big project of the second type as a LilyPond 
infrastructure and one of the first type as a Frescobaldi extension. My idea is 
to have one Frescobaldi extension that provides the core functionality needed 
for both types of repertoires that can be used by a concrete project extension 
(not through some configuration files but by subclassing in Python). 

> 
> For now my priority is to create the command line tools that will meet
> my immediate needs, but if there is a way to do this so that I can then
> more easily reuse the same code as the backend of a Frescobaldi
> extension I will. I'm not familiar with Frescobaldi's code, but I
> imagine that relying on tools provided by python-ly where possible would
> be wise, and that I should generally avoid reimplementing something
> Frescobaldi already does.

Sounds good, if possible.

> 
> For example, the functions I've written so far use `print(line, file=f)`
> to write Lilypond files line by line. See here[2] for instance. I assume
> that Frescobaldi already has a way to write Lilypond files in order to
> make the score wizard work, and that a Frescobaldi extension might be
> better off taking the same approach. Is this something that can be
> abstracted and used outside Frescobaldi in the way that its indentation
> functionality can?

Frescobaldi interacts with the document using Qt's document and cursor 
abstractions, and it partially has its own wrappers around these. You can use 
PyQt5 in command line applications too and could use it to write to your 
documents, but I'm not sure this is really worth the effort.
Probably a more promising approach would be to for example not use print() 
directly but create a wrapper function insert_into_document(text) or similar, 
which can later be replaced with some integration code.

BTW, python-ly was once a part of Frescobaldi and was factored out as a 
standalone repository with all Qt dependencies removed.

> 
> Based on your familiarity with Frescobaldi and Frescobaldi extensions,
> do you have any general advice, or parts of Frescobaldi or that
> extension I should look at? Ideally I would write this as a Frescobaldi
> extension from the beginning, but it's unfamiliar territory for me and
> for now I need to make something usable for my next Lilypond project
> before spending a lot of time learning new skills. In addition to being
> unfamiliar with Frescobaldi, I have never worked with graphical user
> interfaces before.

You might have a look at the files in frescobaldi_app that start with either 
"cursor" or "document" (although not all "document" files are dealing with the 
text document) to get an idea how Frescobaldi handles them.

As for Extensions the two extensions on Github I referenced in the previous 
post give a quite clear introduction into the topic, one is just boilerplate 
code with very little comments, showing the bare necessities for setting up an 
extension while the other is a pretty complete minimal working example with 
lots of explanatory code comments.

Best
Urs

> 
> Thanks,
> 
> Mason
> 
> [1] https://github.com/openlilylib/scholarly/releases
> [2] https://notabug.org/chaosmonk/lilypond-project-manager/src/master/utils.py

Bildschirmfoto vom 2019-08-18 09-30-14.png
Description: PNG image