Re: [Userops] Why is it hard to package libre web applications for distros?

[Olivier Mehani]

Hi Chris, all,

On Fri, May 01, 2015 at 10:13:25PM -0500, Christopher Allan Webber wrote:
> Why indeed?  Discuss!

I think it might be due to developers not trusting systems, and no well
respected standard for file location. Like it or not, at least the LSB
gives an authoritative view of where files should go.

Good old Unix applications tend to come with a list of dependencies to
be provided by the system, and expect to be installed with their files
scattered around LSB-style. This is what the developers has to deal with
as the base.

With web applications, this is entirely different. You don't know where
they will be installed, you don't know if the dependencies will be
available. If they are, you don't know where. And on top of that, you
are not sure that your user will actually have the permissions to fix
the issues that might arise (e.g., Web hosting with no shell access).
The safer way to ship a working application in this case is just to
bundle everything the developer used, and hope for the best.

Most of the time it works, but it's nasty. I was working on a server of
mine yesterday, looking at my list of trusted root certificates, and the
first question I asked was "Which one?" I ended up going for the OS's,
in /etc, but I am pretty sure most web applications, or their
dependencies, will not use it, as they all shipped with a copy. For fun,
I should look at the diffs. So this creates a very inconsistent, and
probably insecure, view of SSL trust across applications.

The same go for utility libraries (I haven't read Chris's post about
this, yet).  Each type ships a different version, in a different
filesystem path (within the application's). What's more, beyond
filesystem path, there is the web path, where the Javascript and other
assets are exposed; once again, there is no standard, and apps look for
their dependency libraries either in
/app/maybe-lib-or-whatever/lib/lib.js or worse (IMO) in a CDN you don't
control. I don't know of a system convention saying "put all libs in
/lib/NAME and all styles in /css/STYLE" (note: all from the root of the
webserver, not per application).

I know Lintian, on Debian, complains when it sees applications shipping
with their own copy of a utility library that has been packaged, but I
think it might be one of the few trying to enforce some sanity. I'm not
sure where Debian installs its system-wide web libraries, nor where it
makes them available. It should be easy to check, though. So, the onus
end up on the packager to do all the leg work to remove the third-party
libraries, and point the application to the system's instead. And it is
painful. And Lintian probably only knows and checks for the most
important ones.

With good old Unix applications, in the vast majority of the cases, you
only get the source for the application itself, and it's up to you (or
you OS) to provide the rest. And the code has a good idea of where to
find it. With web applications, you get all the dependencies, because
it's the only way the developer have some confidence it will work.

So, the problem is two-fold, but mainly stems from a lack of standard
URLs for an applications to load its libraries. The rest of the problem
is the workaround of shipping everything together, making it harder for
the packager to untangle actual application from dependencies.

Another related problem (that I ran into when trying to package
Mediagoblin for OpenBSD) is that of local paths, where a similar issue
arises: where does long-lived data go, or temporary data? Once again, the only 
thing the
application knows is its own app's code, within its installation
directory, creating another entanglement of upstream code and local data
which is 1) hard to update 2) hard to backup 3) hard to mass-host (e.g.,
several instances of the app with unrelated data ran from the same copy
of the code; maybe not with MG, I haven't looked into that yet).

All this to say (as a strawman}, we need webapp-independent path and URL 
conventions:
* long-lived configuration in ${SITE_DATA}/etc/${webapp};
* long-lived data in ${SITE_DATA}/data/${webapp};
* temporary files in ${SITE_DATA}/tmp/${webapp}, or /var/tmp/${SITE}/${webapp};
* dependencies at https://${SITE}/${lib}-${libver}.

As a first, this can be implemented, painstakingly, by the packager.
However, ultimately, I'd say it's the application developer's
responsibility to adhere to this from the ground up. Note that I only
speak of deployed environments, not development ones, though I think one
of the root causes of all this is that webapps tend to be run in
something close to the developer's view, and only mimick this when
deployed.

-- 
Olivier Mehani <address@hidden>
PGP fingerprint: 4435 CF6A 7C8D DD9B E2DE  F5F9 F012 A6E2 98C6 6655
Confidentiality cannot be guaranteed on emails sent or received unencrypted.

pgpTqLC4he0Vx.pgp
Description: PGP signature