Re: packaging lilypond as a docker container?

[Han-Wen Nienhuys]

Let me explain my thinking more in detail. (I've CC'd Jan in case he wants
to give further historical insights)

The objective of GUB was to provide installers for LilyPond with the
following requirements:

1) build for all platforms (OSX, Windows, Linux) with minimal effort,
without having to manage windows and OSX machines.
2) all platforms use exactly the same versions of all software, so everyone
has the same experience, and problems always reproduce across platforms.

1) implied setting up a cross compiling environment. This is a pain in the
ass, because GCC must be compiled for each (host, target) pair separately.
This is very slow and takes up a lot of space.

However, the major reason why this is a PITA is that many pieces of
software don't support cross-compiling well. In particular, some software
runs itself as part of the build (lilypond is an example, but so do Python
and Fontconfig IIRC). Making them cross-compile is something that Jan did
in his professional career (embedded software!), so he knew how to do it
for LilyPond. It is however very intricate, and time consuming to test. One
has to pore over compilation logs, understand what all these packages are
trying to do, and then come up with ways to do the same in a
cross-compiling manner. This is why the GUB specifications are patching
compilation steps in creative ways.

I found it demoralizing to work on, because I want to work on music
typesetting, and not on patching up someone else's wonky compilation script.

Because we now have a very expensive and complicated setup (GCC !), it
makes sense to utilize this setup across builds. For example, if we upgrade
the lilypond version, we'd rather not have to rebuild GCC from scratch. So,
GUB has a packaging system, where you can upgrade LilyPond (or any
intermediate dependency) without starting from scratch.

We started out with a set of shell scripts, but they were hard to maintain,
so we built GUB. So, to me, Jonas' proposal to have a set of simple
compilation scripts is a little funny.

Cross compiling has these problems above, but the "native" compile also has
problems. One has ensure that the native compile doesn't pick up
dependencies from the system it is compiled from, as that could cause
unpredictable behavior. You'd run GUB on a machine that is used for
LilyPond, so all of the dependencies can be consumed from both GUB and the
host machine, and it is easy for something to go wrong here.

In the intervening 15 years, many things have changed.

A negative: apple licensing restrictions say that we can't cross-compile to
OSX (although to be honest, I never read the terms & conditions for the
XCode stuff I downloaded from apple back in the day. Maybe we've always
been in violation of some T&C)

A posiitive: all major platforms run on x86 64-bit CPUs, and there are Open
Source VM solutions for OSX and Windows. So, a VM running Linux on OSX or
Windows can be almost as fast as running Linux directly. This makes GUB's
cross compilation to other architectures superfluous.

A positive: Linux now supports namespaces, and in particular: file system
namespaces. This makes it possible to build a native Linux package fully
isolated from its host system, thus skipping GUB's laborious GCC setup.
This also lets you run a regular package manager on the namespaced file
system, and this is essentially what Docker gives you: a way to install
arbitrary set of dependencies and use them. This makes GUB's package
manager superfluous.

A positive: Docker is open source, and extremely popular technology. Docker
does the hard work of packaging up VM + Docker for windows and OSX as
installer, and providing a management/distribution mechanism for images.

All in all, I think we could use Docker for distributing LilyPond. Then we
would have a distribution mechanism that works for OSX  and Windows, and it
has no overhead to compiling LilyPond as normal Linux package. Because we
can control the base image, we can precisely control the dependencies, and
be sure that the binary behaves the same across all OSes. The best is that
building releases won't require any arcane knowledge of cross-compilation
quirks.

GUIX is Jan's current project. I think it has some similarities to GUB, but
it is focused on providing an environment where all binaries are
reproducibly built from source. This is much more ambitious than GUB, and
seems overkill compared to what we need for LilyPond. I think using it also
entails many more compilation steps, which would makes the release process
slow again.


On Tue, Jan 21, 2020 at 11:28 AM David Kastrup <address@hidden> wrote:

> Jonas Hahnfeld <address@hidden> writes:
>
> > Am Dienstag, den 21.01.2020, 02:38 -0600 schrieb Karlin High:
> >> On 1/21/2020 1:49 AM, Han-Wen Nienhuys wrote:
> >> > if GUB is used, who is maintaining and/or working on it?
> >>
> >> Almost exactly a year ago, there was a sizable "Please test gub" effort
> >> initiated by Knut Petersen.
> >>
> >> <
> >> https://lists.gnu.org/archive/html/lilypond-devel/2019-01/msg00221.html
> >> >
> >>
> >> Clear instructions were given for exactly how to test GUB, and just
> that
> >> thread seems to have involved about 16 people.
> >>
> >> More recently, it looks like Jonas Hahnfeld's work with Python 3
> >> migration involved quite a number of changes to GUB.
> >
> > Yep, and it's not been a very pleasant experience. To be precise here:
> > It's not about porting GUB to Python 3, it's just that porting LilyPond
> > will introduce a dependency on an updated package for the next release.
> > So while David is mostly correct that contributors don't need to bother
> > with GUB, that doesn't hold true once you want to bump one of its
> > dependencies...
> >
> > Based on this experience, I've thought about how to improve the process
> > of building binary releases for LilyPond. What I have been
> > experimenting with is a set of portable shell scripts that build mostly
> > static executables. I've a prototype ready at
> > https://github.com/hahnjo/lilypond-binaries which works for Linux and
> > FreeBSD. I don't see a reason it cannot work on macOS, but I don't have
> > the possibility to test...
>
> The problem is "on macOS" since current macOSX library/toolbox licenses
> all prohibit use in cross-building environments.  It looks like a given
> that we will not be able to offer macOSX libraries in future as the
> result of a unified release process.  Our sources build reasonably well
> that macOSX integrators using some of the macOSX typical packaging
> systems are able to provide reasonable replacements.
>
> But the elephant in the room is Windows.  Han-Wen says he never wants to
> touch GUB again (and he actually didn't as far as I remember) but I
> don't want to touch Windows again, and GUB provides them.  The
> dependency nightmare on a non-UNIX like platform was what brought GUB
> into being in the first place: native builds are much more of an ongoing
> problem.  Just look at the continuous effort the Git project has in
> keeping a Windows version available.
>
> >>  From the notes I've kept, Jan Nieuwenhuizen proposed replacing GUB
> with
> >> something based on Guix.
> >>
> >> <
> >> https://lists.gnu.org/archive/html/lilypond-user/2016-09/msg00690.html
> >> >
> >>
> >> That idea was mentioned a few times since, but I can't recall if
> >> GUB-replacement discussions have moved beyond that or not.
> >
> > As far as I understand, Guix is a package manager in the first place.
> > In the thread Jan proposes to use it for cross-compilation, which I
> > think is the primary reason for the complexity in GUB. So why go that
> > route once more?
> > Instead I designed the mentioned scripts such that they produce native
> > executables. That's also how TeXlive builds their packages, for
> > example, and it works quite well if you compile on the oldest OS that
> > you intend to support (CentOS 7, FreeBSD 11).
> >
> > I haven't decided yet how to compile for Windows. Maybe that's still a
> > valid use case for cross-compilation (but only with a very limited
> > scope).
>
> Windows really is the elephant in the room.  MacOSX will cater with
> native port systems like MacPorts etc and other UNIX-like systems also
> have working packagers and package systems.
>
> > If this attempt sounds interesting to the community, I would be happy
> > to submit the scripts for inclusion into the LilyPond repository
> > itself. That would also solve another issue with GUB: Currently it's a
> > separate repository with no way to keep it in sync with changing
> > dependencies for LilyPond...
>
> Maybe we should entertain two branches of GUB matching current stable
> and unstable release tracks?  Or otherwise deal with a difference in
> dependencies for stable/unstable?
>
> --
> David Kastrup
>


-- 
Han-Wen Nienhuys - address@hidden - http://www.xs4all.nl/~hanwen