Salzburg resume

[Han-Wen Nienhuys]

the event in Salzburg has terminated. Full notes are here, but I'll post
hte contents inline for those without google docs access
https://docs.google.com/document/d/1Xj_31AZXxwWOrP1zv7Itjx_9aQshBVshWc58gBdkICA/edit#

Conference materials: https://gitlab.com/MusicEngravingConference/2020

JP Voigt - data exchange talk


   -

   (slides)
   -

   Export to MEI preferable over MusicXML
   -

   Export to XML (wolff vs JP’s)



   -

   HW:  lilypond aligns to time stamps; how does that work with parsing
   based solutions? WB: it doesnt, you get separate streams. There is also SCM
   code that exports all lilypond code time aligned, but still expressions, so
   no positioning
   -

   DK: we could have 3rd output option, next to midi and paper.
   -

   JPV: didn’t want to go into the C++ code destroying it. Prefer to use
   OLL so it can be used with any lp version.
   -

   UL: do we want to preserve layout info? The important use case is to not
   care about layout, and this is the easy case.
   -

   JPV: presented this in 2017
   -

   WB, JPV: music expression is not executed (?)
   -

   HW: does anyone use ab2ly, etf2ly etc?
   -

   JPV: integrate import into lilypond, then you can have do point and
   click pointing back to XML.
   -

   DAK: ABC is still used by the huge corpus.
   -

   UL: it is mentioned if it doesn’t work
   -

   JM: happy customers do not complain.
   -

   UL: how can we avoid depending on JPV doing preso.
   -

   JW: needs a clear vision. Every approach has tradeoffs.
   -

   UL: I only do something if I need it, or if contributor comes. IF we
   have a place where we can direct contributors, that would help
   -

   MS: email is not my primary communication. Slack, Whatsapp
   -

   UL: younger generations not on the mailing list. Diversify onto
   different platforms is fragmenting.
   -

   JW: different channels, for different purposes, we could redirect to
   different places. Eg. all questions to stackexchange. Emails can more
   easily lead to flamewar/long discussions. Waste of time. Slack has the
   benefit of real-time.
   -

   JPV: development discussions
   -

   MS: on functional programming site, scheme questions are the place.
   Creating something is cheap. If it gets traction, nobody cares afterward.
   -

   JW: re. Slack. Concerns about communication/format, what about heritage?
   Slack is good for ad-hoc comms. Anything that needs to be preserved should
   go somewhere else, eg on mailing list.
   -

   HW: slack yet another source of interruption
   -

   AK : only works for people that are on their screen the whole time.
   -

   DK: active developers are too busy to keep practicing.
   -

   JPV: competing seeds to do something; some can coexist.
   -

   DK: lack of development time & power. Abc2ly still seems to work.
   -

   DL: tried another one. ABC files with many scores don’t work.


Decisions:

   -

   Mike proposes to try out an alternate forum? (stackoverflow)?


Urs Liska - 10 min, openlilylib

   -


Jacques Menu - xml2ly

   -

   Slides (TODO: link)
   -

   Libmusicxml2
   -

   https://github.com/grame-cncm
   -

   MSR (Music Score Repr) from the xml representation of libmusicxml2
   -

   LPSR (LilyPond Score Repr) has the containers (score, book etc.)
   -

   Discussion on how to walk the tree with STL.
   -

   Lots of option to automatically adapt outputs (transposition, Jianpu /
   unusual notation styles, augmentation with headers, instrumentName etc.)
   -

   Different from musicxml2ly it *does* solve issue 34 automatically
   (graces)
   -

   Related projects: xml2brl (Braille output), xml2guido
   -

      Nobody interested; but picking up again with new contacts at MEC
      -

   To convert from LilyPond to other formats you need to stop LilyPond at
   some intermediate point where you have a scheme representation of the music
   and convert it to LPSR; from there you can convert to MSR and then to other
   formats
   Question: what is that intermediate point? Where do we stop LilyPond?
   -

   Inside the lilypond, doc. (link?)
   -

   How does the text go from notes to objects
   -

   HW: I can explain it, but there it takes time and i need a white board.
   -

      JF: I also want to know this.
      -

   JPV: seems to work pretty well. Could we integrate it in LilyPond? Need
   to have an internal score representation
   -

   UL: can you skip the lilypond input step?
   -

   HW: how would you apply tweaks?
   -

   UL: not the only way?
   -

   HW: have to maintain two ways.
   -

   UL: want to run directly on MEI input
   -

   HW: partial solution in Erik Sandberg’s thesis that describes handling
   events on-the-fly
   -

   UL: want to have lilypond as tool in the MEI comm, run directly on MEI
   data instead, but speed is a concern.
   -

   JM: xml2ly could be adapted to mei2ly by converting MEI to the MSR and
   using the already existing outputs to lily and braille
   -

   HW: strange that cpu cost of conversion would be a factor.  You could
   just shell out to xml2ly and run on the output in the background.
   -

   JM: xml2ly c++ O(10s) for beethoven symphony.
   -

   JF: parsing and midi is nearly free.
   -

   JPV: XML is a data storage, ly file is executable.
   -

   JM: thanks to Werner


Thomas Morley: Lilypond-Erweiterungen aus Benutzerperspektive

   -

   Slides (todo: link)
   -

   No developer, no programming background
   -

   Manuals are actually not that bad for Scheme and Guile, if you work
   through them
   -

   Arpeggio arrow. How to override stencil
   -

   Commands to dig into implementation: \display-lily-music, (pretty-print
   stencil-expr)
   -

   Step by step demo on how to extend arpeggio
   -

   What did we use? GUILE/lilypond elements used.
   -

   Well documented in green (very short list)
   -

   Stencils mixed bag. Intervals undocumented.
   -

   This was simple, no mapping, no recursion, yet a lot of a poorly
   documented things.
   -

   As a user, try out fragment I found at random. Not very efficient.
   -

   Wish for “General Coding Reference”, eg. “interval calculation”.
   -

   No index for SCM files. 10000 lines with markup, too much to read.
   -

   WL: lots of functions well documented
   -

   UL: most doc strings are mostly useless.
   -

   DK: lilypond doc is “someone else’s problem.” - beginners don’t know
   where to start?
   -

   HW: can we make frescobaldi the official editor? DK: there is no another
   one
   -

   DK: used to be delivered with lilypad? What will user aspiring user work
   with? Tutorial that focuses on Frescobaldi is more useful.
   -

   TM: user that wants to extend it; I don’t like Frescobaldi.
   -

   DK: doc should be usable outside of Frescobaldi.
   -

   DK: if you want to go deeper, the doc is not there, and if you are a
   beginner the doc is not there.
   -

   TM: have to use the search function of the editor.
   -

   UL: Make frescobaldi download lilypond.
   -

   HW: Does it run on windows?
   -

   WB: v1 did have it.
   -

   HW: is windows still a thing?
   -

   DK: 2 integrators come from windows, and use on windows privately. Don’t
   underestimate.
   -

   MS: microsoft conf, vscode devs. Telemetry is useful; spooky.
   Empathising, we don’t correspond to our users. Telemetry is hairy (GDPR?),
   but can help empathy.
   -

   MS: VScode has a “sos” button that sends a ping to a relevant forum.
   -

   UL: where would that go? Post to the mailing list?
   -

   MS: “im available on my commute home”
   -

   UL: needs a platform
   -

   MS: wanted to bring it up.
   -

   JW: out of time.


Conclusions:

   -

   Need more better and more documentation for internals.


Mike Solomon

   -

   Political/social aspect of coding.
   -

   Discuss background of pure containers.
   -

   How can we nurture people that are beginners
   -

   Pure/unpure containers.
   -

   Circular dependencies.
   -

   (example on blackboard)
   -

   Make estimates in single pass.
   -

   Skylines are used for vertical alignment
   -

   Eliminate circular dependencies
   -

   Constraint optimization can’t work in lilypond is too expensive
   -

   Cross staff beam example.
   -

   X-staff beam influences vertical skyline, which influences line
   breaking, which influences beam
   -

   Acknowledge that personal pleasure is a big part of working on LilyPond
   -

   Influences day to day decisions: do I make time to work on LilyPond
   -

   Proposed late in 2.18, stressed out people doing release management.
   -

   Worked in branch, branch drifted too much
   -

   James Joyce: Finnegan’s wake: why is it soo hard to read? Becaus it was
   hard to write.
   -

   being clever
   -

   No choice to opt-out out of unpure/pure stuff.
   -

   How can we nurture more ambitious changes; I’m a binge-coder, so can’t
   do incremental changes?
   -

   How can we make it more appealing to more radical changes, or should we
   not be?
   -

   WL: is there any chance you’ll invest time again into lilypond coding?
   -

   MS: yes? CEO of a company, busy. Nobody has time.
   -

   MS: yes, in the long term.
   -

   MS: in the short term, if there were communication channels. More
   appealing to me, more interactive.
   -

   WL: only reachable through Whatsapp.
   -

   MS: my email goes through an assistant, to shield me from bullshit.
   -

   DK: simple-closures. Opaque in use, undocumented. Removed, and
   pure/unpure was an improvement.
   -

   DK: circular deps, could we figure them out dynamically? Unpure get
   left/right column. Have a single callback, and lilypond could decline to
   give answer to avoid circular deps.
   -

   TM: pure/unpure not documented well.
   -

   DK: need documentation on when they be called, and how they will.
   -

   HW: want to get sucked in.
   -

   WL: github mirror, daily pull.
   -

   WL: offer: if the mirror is up to date, make a PR, we’ll make it do.
   -

   WL: changing the system (gerrit, github), need a person to do
   -

   KM: zones of genius. Don’t have a huge pool, but this is how we going to
   grow. Different people have different barriers. HW no patience for process,
   KM needs more understanding.
   -

   WL: I’m a good assistant; I’m not a genius contributor.
   -

   DK: Kieren is power user, very active, not very commits in main repo. If
   Kieren does not find a way to contribute properly, then something is wrong.
   -

   JW: Make contributing easier, I want to do it. Do we agree on how we
   want to do it.
   -

   WL: modern way contributing = git repo with pull-requests
   -

   WL: can squash?
   -

   KM: this is the reason. I solve it for myself, but process seems an
   obstacle to me.
   -

   WL: writing the code is 20%, writing doc, etc. is 80%
   -

   KM: I’m not the person to doc & push to github, I need someone to
   shepherd it through the process.
   -

   DK: we need a loosely associated code based, turnkey ready, like LSR.
   Needs to have quality control.


UL - openlilylib

   -

   (slides)
   -

   Snippets - dump of stuff, more or less deprecated
   -

      Moved some snippets to oll-misc
      -

   Current use cases: both packages and building blocks
   -

   Not sure if wrappers/syntactic sugar is a good idea, or whether the
   plain functionality should be better documented (first)
   -

   Proposals:
   -

      Package manuals for packages
      -

      Code documentation
      -

      Frescobaldi integration
      -

   How can OLL be a place for extension infrastructure
   -

   HW: npm / pip / rust-like package manager is today’s approach
   -

   MN: can already do
   -

   HW: how do you find the packages, where do you find the doc.
   -

   DK: have a hierarchy of packages that are on local disk. Ferneyhough
   hairpins. First drop in your own package, then write doc, then migrate to
   OLL. Then migrate into lilypond.
   -

   UL: Need to move the package loading/handling to lilypond core itself.
   -

   UL: large code, written by me. Coding quality.
   -

   JW: take a step back. Do we agree that we want OLL to make the package
   management for lilypond?
   -

   DK: I don’t know how it works, UL: I’ll describe what to look at.  I
   need strict code review, and it is a large addition.
   -

   DK: quality of the design is more important than code quality, the
   latter can be fixed later.
   -

   UL: will discuss ideas/design choices on -devel.
   -

   MN: can experiment with things, want to recommend trying it out.


Decision:

   -

   Urs Liska to forward relevant docs to David K for evaluation.


Wilbert - Frescobaldi

   -

   (slides)
   -

   Assembler on MX spectrum.
   -

   From MUP to lilypond. “lily was here”
   -

   Need an editor; started as KDE plugin lilyKDE,
   -

   grew to standalone app Frescobaldi in 200{7,8}; v1.0 Xmas 2009
   -

   Recycled frescobaldi from XML standard and domain name.
   -

   KDE4 - no plugins anymore => full rewrite with Qt4 for 1.9.0
   -

   Python-ly became separate
   -

   Edit features: add trill to note by clicking on note & trill.
   -

   Frescobaldi can parse HTML, Lilypond, latex and understand embedding.
   -

   Tree structure;
   -

   Frescobaldi understands the music tree (eg. length of music)
   -

   Parser has been reused in Peter’s musicxml converter
   -

   HW: super impressed that you parse the input.
   -

   UL: learned python through new code base.
   -

   JW: librarian, distributed engraving. Frescobaldi very useful for
   getting newbie.
   -

   WB: songbook of protestant church. Dutch songbook. 300k prints, 1600
   pages. Organ accompaniments. Structure with includable files. Took 1.5
   years.
   -

   KM: using lilypad, mind blowing to use Frescobaldi.
   -

   WB: lilypondtool by bertalan fodor. Stole a lot of ideas.
   -

   KM: should new users be steered to Frescobaldi
   -

   WB: everyone can pick what they write
   -

   DK: recommend Frescobaldi for new users; let download page point to
   frescobaldi
   -

   UL: don’t have to make it integrated.
   -

   HW: don’t integrate, because both projects will hamper each other.


Conclusion:

   -

   Website/documentation should encourage Frescobaldi for new users.


Lunch break till 14:30.

Kieren MacMillan: Edition Engraver

   -

   Move “presentational control” to external area independent of musical
   content
   -

   Set final barline in edition engraver, because piece used in different
   contexts.
   -

   inject changes at specific moments
   -

      TODO: inject changes at position x after some tag
      -

   (something about dots and identifiers)
   -

   Small question: is that nice to have? => Agreement by everybody.
   -

      HW, JR: that was the original intention
      -

   Big question: is the edition engraver the right tool for it?
   -

   Multiple moments with same mod, multiple mods on same mods.
   -

   DK: have a way to switch on a feature.
   -

   HW: why is adding an engraver not an extension?
   -

   KM: polymetric example #(allow-polymetric) is much more intuitive than
   modifying layout block.
   -

   MS: syntactic sugar? KM, UL: yes!
   -

   MS: having edition engraver in OLL is a great direction. Maybe we should
   rather move other “core engravers” to softcoded in OLL
   -

   DK: but does that solve anything?
   -

   KM: “after mark C, 1 measure afterward”:
   -

   DK: “before mark C” ? Write out timing information in separate part.
   -

   HW: tweak rehearsal mark, keeps track of timing information since
   rehearsal mark. Much simpler than a 2 pass system.
   -

   KM: should not be rehearsal mark, but anchor mark.
   -

   DE: mark engraver, dC/dS
   -

   KM: Code management, many mods, organizing mod code is not obvious.
   -

   KM: coding standards
   -

   Conclusion:
   -

      Separation is great
      -

      Keep in OLL is great
      -

      Could implement anchors
      -

      Change staff injection causes crashes
      -

      Polymetric music hard to deal.  Anchors would solve all of it.


Davide Liessi - Frescobaldi for Mac

   -

   (slides)
   -

   Debugging Qt reducing. Took a whole day.
   -

   KM: started using after it became an app bundle.
   -

   WB: debug from command line.
   -

   WB: open for hardware donations for Mac.
   -

   DL: need a new Mac. Some problems are with recent versions.
   -

   KM: offer. Colocated devices (mac minis)? Remote desktop. LilyPond build.
   -

   DK: does this work?
   -

   HW: oldest mac you can use for this. DL: Lion, 10.7, Latest 2-3 versions
   of 10.13 - 10.15 (Qt). 2013 machine is new enough. everymac.com
   -

   MN: MIDI not working … cannot help with this
   -

   Behind bars. Gould standard of engraving. How well does Lilypond do
   follow Gould’s standard? Collab project to re-engrave examples, to see
   where lilypond stands relative to the standard. Fake tempo spanners with
   Lilypond, perfectly align with other tempo markers.
   -

   UL: publisher want specialized price with 5 yr license for specific
   examples. Will explore.


––––––––––

Han-Wen & Janek - Topics for discussion

Development Process + Janek’s involvement

   -

   JW: friction in process is the main reason for my inactivity
   -

   Changing tools & processes
   -

   Make contributing more accessible to outsider community
   -

   JW committing to going back to work on process, but want buy-in from
   community.
   -

   Confident that I can make the process friendly to both Han-Wen & Kieren,
   then it will work for the general public.
   -

   KM: I buy in.
   -

   HW: +1
   -

   JPV: +1
   -

   JR: +1
   -

   DK: I can’t buy into what I don’t know
   -

   JW: If there is no consensus, can I be the benevolent dictator of
   development process??
   -

   KM: what if DK doesn’t say yes now, will there be a moment you can’t do
   your job.
   -

   JW: I want lilypond on github, can still be hosted on savannah.
   Pull-Requests should be first class
   -

   DK: first class way via Github. GNU resources.
   -

   HW: I want review integrated with hosting, and I want it to be hosted.
   -

   UL: hosting too impractical.
   -

   HW: GNU doesn’t allow sourceforge; haven’t kicked us out yet.
   -

   JW: Would you (vote!) want to move to Github if we had to leave GNU?
   -

   HW: I care about the users, so happy forego to GNU.
   -

   DK: debian based packages guile based work.
   -

   DK: GNU rather than github, more reliable.
   -

   DL: macports supported by apple; closed down by Apple. Main repo in
   github, bugtracker in self-hosted Trac. Separated. Bots doing the back &
   forth.
   -

   DK: current setup sucks.
   -

   DK: put in server controlled by microsoft.
   -

   DL: Only on Microsoft server.
   -

   DK: If we have bug tracking system, need to be able to export bug
   tracking system.
   -

   HW: does sf.net have export?
   -

   UL: github has an API.
   -

   HW: GNU is volunteer run; GH/GL are professional companies with an
   incentive to keep things working. GL is VC backed, if they run out of
   money, no diff.
   -

   KM: actionable.
   -

   JW: Getting more contributors
   -

   UL: I trust Janek.
   -

   DE: I love a better process, too much friction today.
   -

   KM: I trust you!
   -

   JPV: I would love just using a PR
   -

   HW: I trust you; need to think about CI.
   -

   DK: part of test suite is visual comparison
   -

   HW: should be part of code review
   -

   T.M: don’t like Github, b/c of microsoft. I trust JW
   -

   HW: can we do a design/requirements doc, and then it follows what our
   options are and how to proceed.
   -

   DK: get bug tracker data out
   -

   HW: should be part of requirements doc.
   -

   DL: extract metadata out of GL/GH is possible.
   -

   MN: trust you; we should work out the contribution process, not focus on
   infrastructure.
   -

   MS: [remote] I will make a PR for a code of conduct.

Release of 2.20, 2.21, etc.

   -

   DK: Tasks for release:
   -

      Commits from unstable to stable?
      -

      Conference preps
      -

      No change in binary platforms. 64 bit MacOS comes from macports.
      -

      We will deliver 32bit OSX and powerpc.
      -

      Remove PPC OSX after this release.
      -

   HW: so just backports if we want.
   -

   DK: Need another prerelease, have to ask Phil. In principle 2 weeks.
   Just like 1 month ago.
   -

   JW: release what we have as 2.20?
   -

   DK: want to forget about 2.20 as soon as it is out.
   -

   KM: so let’s do it!
   -

   DK: decision: do 2.19.84 now, until next weekend. 2 wk period (time
   boxed) for cherry-picks, then release 2.20.


Guile 1.8 vs. 2 & 3

   -

   HW: pretend our UTF8 is Latin1, then it should work.
   -

   KM: downside [for user]?
   -

   HW: have to convert to and from 32-bit representation in Scheme code, if
   they want to manipulate Unicode code points from Scheme. Unicode isn’t
   supported in Guile 1.8 either, so no big loss.
   -

   DK: we can try this and see the impact.
   -

   DK: Guile1 is an interpreter. Guile2 compiles into an intermediate
   representation. Don’t know performance impact.
   -

   JF:  ?
   -

   KM: GUILE developer?
   -

   DK: one developer Andy Wingo working on compiler.
   -

   DK: political, Andy signed manifesto against RMS.
   -

   HW: so did we.
   -

   DK: guile2 is doable; haven’t looked into running GUILE v2.
   -

   TM: we have a branch for guile2 work. Last try was successful.
   -

   TM: full char-set in guile-1: 256

TM: full-char-set in guile-2.9: 1112064

TM: makes operations much easier in guile-2

TM: huge slowdown. Precompiled possibilities.

   -

   DK: byte compiler must be written to disk. Have to store them somewhere
   -

   HW: could do bytecode compilation as part of lilypond compilation.
   -

   DK: need to precompile using dependencies, which are provided by
   LilyPond.
   -

   DK: need to precompile and install
   -

   JW: can we cut corners?
   -

   DK: we can cut corners by cutting performance in half.
   -

   DK: slowdown comes from somewhere.
   -

   DK: fixed several examples by providing crashing C++ code.
   -

   JF: Ubuntu 18, guile not standard available. Does this inhibit.
   -

   DK: Lilypond includes Guile
   -

   DK: no final decision on Guile; TM is the person that knows best.
   -

   TM: test latin1 encoding.


UL: Summary of this, of decisions and resolutions considered and/or agreed
upon should be compiled and sent to lilypond-devel

DE = Dan Eble

KM = Kieren MacMillan

MS = Mike Solomon

JW = Janek Warchoł

UL = Urs Liska

HW = Han-Wen Nienhuys

JM = Jacques Menu

WB = Wilbert Berendsen

AK = Alexander Kobel

DL = Davide Liessi

DK = David Kastrup

JF = Johannes Feulner (from Scorio)

JR = Jürgen Reuter

TM = Thomas Morley (Harm)

MN = Martin Neubauer

JPV = Jan-Peter Voigt

WL = Werner Lemberg

UL (remotely): Just having a look at that name list above is a reminder of
how exceptional this opportunity just has been!!! Thank you all for coming.
Topics for discussion

(please list what you want to have discussed. To keep the list focused
please suggest no more than 2-3 items.)


   -

   Development process
   -

      Code hosting / code review tooling (hanwen, janek, +1)
      -

      Communication (and communication tools) (+1)
      -

         Mattermost/Slack/gitter vs email
         -

         Communities like dev.to
         -

         Offline events (ie meetup.com)
         -

         More regular development meetings? How to organize?
         -

         Decision-making dynamics, avoiding inertia
         -

      Making it easier to contribute (+1, hanwen, janek)
      -

         No single process for all developers
         -

         Getting someone excited (especially someone like MS and HW) is
         crucial
         -

         Ways for more accessible documentation of the internals
         -

         Update/write code to make the documentation already available in
         scheme functions appear in the internal reference.  Right
now, only a small
         subset gets extracted
         -

         Removing obsolete information from Contributors’ Guide
         -

   LilyPond core directions:
   -

      Release 2.20
      -

      Guile 1.8 fork vs 3.0? (+1)
      -

      What extension points needs to be added?
      -

      Increasing performance (+1)
      -

   JW: Lowering the barrier to entry for new users
   -

      Making Frescobaldi the official editor
      -

      Smoother installation experience (one-click installer for everything)
      -

      How to make it easier for new users to get help? In-Frescobaldi help?
      -

   Davide’s Behind Bars project (nice but later) (+1)
   -

   JW: Style guide for lilypond files (and an official linter?) - not
   important enough for today
   -

   Managing power and complexity: (+1)
   -

      With custom infrastructures one can achieve “everything”
      -

      These are reserved for a small number of power users
      -

      However, it’s possible to *provide* complex infrastructures and have
      less fluent users *use* it for work.
      -

      => Could we create a way for making this more accessible, known and
      appealing to attract a wider user base?
      -

   openLilyLib’s place in the ecosystem (prepared in scheduled talk)
   -

      direction of development
      -

      Community engagement
      -

      [DL: more generally, how do we want to deal with "extensions"? if we
      want to follow the LaTeX example, we need a package collection & manager;
      what about lyp (http://lyp.noteflakes.com/)?]
      => One idea for that might be the planned Frescobaldi extension for
      managing and installing openLilyLib (packages).
      -

   How does the backend work (“whiteboard presentation.”)
   -

   Ways to integrate parallel development (in secondary tools like
   templating systems)
   -

   Improving the slur algorithm: how to get debug output, how to see the
   scoring system in action (together with the demerits), some special code
   that prints the considered slurs step by step, etc.



-- 
Han-Wen Nienhuys - address@hidden - http://www.xs4all.nl/~hanwen