Re: [Userops] Why is it hard to move from one machine to another? An analysis.

[韋嘉誠]

On Apr 8, 2015 6:20 PM, "Christopher Allan Webber"
<address@hidden> wrote:

>    - /var/ is mutable, it's data.  There's no making this "reproducible"
>      really: it needs to be backed up and moved around.

Yup. But it's a bit flakey. Package managers write there too, not just
sysadmins and installed software itself.

>    - My packages and system are immutable, or mostly should be.  Even if
>      not using a perfectly immutable system like guix/nix, it's helpful
>      to *act* like this part of the system is pseudo-immutable, and
>      simply derived from some listing of things I said I wanted
>      installed.  (Nix/Guix probably do this the most nicely though.)

Absolutely.

>    - /etc/ is similarly "immutable but derived" in the best case.  I
>      should be able to give the same system configuration inputs and
>      always get the same system of packages and configuration files.

This is like /var but in the other direction. Mostly package
territory, but you need to edit things. Horrible to reproduce.

Would be nice if all of it worked like /etc/default and /etc/sysconfig
on not-Debian: Packages write their stuff to default, you override in
sysconfig. Hypothetically you could move sysconfig to a fresh machine
and things would work. No do this for all of /etc for all packages.
Not a trivial undertaking but maybe software makers should keep it in
mind and start doing it.

>  - I like Guix/Nix, but my usage of Debian and Fedora and friends is not
>    going away anytime soon.  Nonetheless, configuration management
>    systems like puppet/ansible/salt help give the *illusion* of an
>    immutable system derived from a set of inputs, even though they are
>    working within a mutable one.

Yeah. At least they make /etc derived from rules.

>  - Language packaging for deployment needs to die.  Yes, I say this as a
>    project that advocates that very route.  We're doing it wrong, and
>    I want to change it.
>    (Language packaging for development though: that's great!)

After reading later responses I realized this means
virtualenv/npm/bundler, programming language-specific
bundling-in-one-directory of dependencies and probably state.

>  - Asking people to use systems like ansible/salt/puppet is asking users
>    too much.  You're just asking them to learn one more layer on top of
>    knowing how the whole system works.  Sharing common code is mostly
>    copy and paste.  There are some layers built on top of here to
>    mitigate this but afaict they aren't really good, not good enough.
>    (I am working on something to solve this...)

Yeah, definitely not userops, quite the opposite side of the
sysadmin-userops scale.

>  - Pre-built containers are not the solution.  Sorry container people!
>    Containers can be really useful but only if they are built in some
>    reproducible way.  But very few people using Docker and etc seem to
>    be doing this.

Isn't that what Docker files are for? Are people doing course work
with them and then ssh:ing in and handmaking their special snowflakes?
Horrid.

> But here's another thing: Docker and friends contain
>    their own deployment domain specific languages, which is dumb.

But isn't that what you are making too?  What's wrong with them and
how is yours fixing it?

>    maybe we can use containers as lightweight and even sandboxed VMs,
>    but we shouldn't be installing prebuilt containers on our servers
>    alone as a system.

Yeah, containers should be more like a polyglot virtualenv and less
like a system.

>    Otherwise else you're running 80 heavy and expensive Docker images
>    that slowly go out of date... now you're not maintaining 1
>    distribution install, you're maintaining 81 of them.  Yikes!  Good
>    luck with the next Shellshock!

Mmm. But if people built them from Dockerfiles it would just be a
matter of rebuilding, right?

>  - Before Asheesh jumps in here: yes I will say that Sandstorm is taking
>    maybe the best route as in terms of a system that uses containers
>    heavily (and unlike Docker, they seem actually sandboxed) in that it
>    seems to have a separation between mutable parts and immutable parts:

Yeah, I like that you are making that the core of your argument. It's
absolutely about separating upstream code and data from data for my
instance.

>    the container is more or less an immutable machine from what I can
>    tell that has /var/ mounted *into* it, which is a pretty good route.

Cool. I guess apps need to be adapted to make it work cleanly.

> So there are some reflections in case you are planning on debugging why
> these things are hard.

I think part of why desktops are easier than servers is because we
have made them more like fat clients and moved all the data to
servers. But the immutable system vs user data kept in one place is
definitely part of the story too.

Because services are not run as several instances by several users on
one machine, their state has been allowed to nestle itself into
several places on the system.

This is why bundler/npm/virtualenv is often the best way to deploy,
because it isolates the application state (and dependencies) from the
system (and other installed services). Docker looked like it could be
a polyglot version of this, but I guess from what you are saying
people are not using it like that.


As some other responses mentioned, eventually state needs to go
somewhere. If we make our neat, non-bloated, reproducible Docker
containers or polyglot virtualenvs, and we push all state into a
database somewhere, still that database needs to be upgraded at some
point. And Docker doesn't solve that. We need to think about that
mutable/immutable separation whatever we do. But I think keeping it
minimal in each service and having one state-service like a database
taking the most of the burden is probably a good idea. Microservices
people seem to be into this, and I think they're right.