Re: packaging lilypond as a docker container?

[David Kastrup]

Han-Wen Nienhuys <address@hidden> writes:

> Let me explain my thinking more in detail. (I've CC'd Jan in case he wants
> to give further historical insights)
>
> The objective of GUB was to provide installers for LilyPond with the
> following requirements:
>
> 1) build for all platforms (OSX, Windows, Linux) with minimal effort,
> without having to manage windows and OSX machines.
> 2) all platforms use exactly the same versions of all software, so everyone
> has the same experience, and problems always reproduce across platforms.
>
> 1) implied setting up a cross compiling environment. This is a pain in the
> ass, because GCC must be compiled for each (host, target) pair separately.
> This is very slow and takes up a lot of space.

[Skipping over a number of details]

> In the intervening 15 years, many things have changed.
>
> A negative: apple licensing restrictions say that we can't cross-compile to
> OSX (although to be honest, I never read the terms & conditions for the
> XCode stuff I downloaded from apple back in the day. Maybe we've always
> been in violation of some T&C)

I think the "no crosscompilation" thing was some Jobs vendetta against
something specific he didn't like, and was introduced to thwart one
thing in particular.  I don't remember the details, but it's possible
that the ancient XCode is from before that event.  We have tried a bit
of finding out but unsuccessfully so.

A possibility would be to go to whatever became of OpenDarwin and use
that, but as far as I remember some of the font handling libraries in
use were actually tied to MacOSX in particular (Cocoa by now?).

> A posiitive: all major platforms run on x86 64-bit CPUs, and there are
> Open Source VM solutions for OSX and Windows. So, a VM running Linux
> on OSX or Windows can be almost as fast as running Linux
> directly. This makes GUB's cross compilation to other architectures
> superfluous.

In ancient times, I've had to be the guinea pig for VMware in a large
company and the experience was not thrilling: two of the typical
problems were that not the full power of a 2-CPU computer was made
available to Linux, only part of the RAM (and the basic RAM was getting
managed by Windows) and mapping the file system through a Windows file
system was really bad since Linux is so much faster in that regard.
That particularly concerned Git use which really depends on fast file
system operations.  Creating a virtual disk helped somewhat, creating an
actual partition for Linux helped quite more.  At some point of time I
started booting that partition, and, well, the experiment did not
deliver a lot more relevant data afterwards...

I have no experience with Docker and containers.  That was a full VM at
the time.

> A positive: Linux now supports namespaces, and in particular: file
> system namespaces. This makes it possible to build a native Linux
> package fully isolated from its host system, thus skipping GUB's
> laborious GCC setup.  This also lets you run a regular package manager
> on the namespaced file system, and this is essentially what Docker
> gives you: a way to install arbitrary set of dependencies and use
> them. This makes GUB's package manager superfluous.
>
> A positive: Docker is open source, and extremely popular
> technology. Docker does the hard work of packaging up VM + Docker for
> windows and OSX as installer, and providing a management/distribution
> mechanism for images.
>
> All in all, I think we could use Docker for distributing
> LilyPond. Then we would have a distribution mechanism that works for
> OSX and Windows, and it has no overhead to compiling LilyPond as
> normal Linux package. Because we can control the base image, we can
> precisely control the dependencies, and be sure that the binary
> behaves the same across all OSes. The best is that building releases
> won't require any arcane knowledge of cross-compilation quirks.

It suffers from the "someone needs to do it" syndrome and I have
absolutely no idea how smooth such executables will install and behave
compared to native ones on MacOSX and Windows.  I'd have to rely on
evaluation by the respective OS users to have an idea about that.

> GUIX is Jan's current project. I think it has some similarities to
> GUB, but it is focused on providing an environment where all binaries
> are reproducibly built from source. This is much more ambitious than
> GUB, and seems overkill compared to what we need for LilyPond. I think
> using it also entails many more compilation steps, which would makes
> the release process slow again.

I don't think it has an answer for the elephant in the room: Windows.

-- 
David Kastrup