Re: It's time to build "guix deploy"

[Thompson, David]

On Tue, Feb 12, 2019 at 8:34 AM Christopher Lemmer Webber
<address@hidden> wrote:
>
> Thompson, David writes:
>
> >> Maybe Dave and I can meet up IRL now that we're close enough to each
> >> other to chat about it.  But I know it's less fun than it used to be for
> >> Dave to consider this because now that's Dave's actual job... but all
> >> the more reason we need Dave's wisdom! :)
> >
> > We could meet up IRL about this and I can try to make an earnest
> > effort to deal with this. I think what has stopped me in the past is
> > the sheer size of this project, and maybe dramatically scaling down
> > the scope will allow us to get *something* out the door.
>
> Yes I think we shouldn't allow ourselves to get too overwhelmed!  We
> should give the warnings that this is an alpha system in alpha (soon
> beta?) software and we don't guarantee that the api will be stable :)
>
> Furthermore, I think if we put an emphasis on composable layers, and
> start with the simplest lower layers, and then add some other layers
> above it (and actually the different use cases you lay out may actually
> swap out which layers are above this), then I think we could be good?

Something like that, yeah.

> > Here are some general use-cases I know about for deployments, roughly
> > ordered from small scale to large scale, and least complex to most
> > complex:
> >
> > * Managing a physical machine or two that have been given memorable
> > names that you update in-place (home scale)
> > * Managing a virtual machine or two that have been given memorable
> > names that you update in-place (blog scale)
>
> In the past we have discussed storing "deployment state", and where and
> how we would store it.  I will make the argument that for these two
> cases, we actually don't need state!  We can just do the simplest thing
> possible: build the system derivation for each of these machines, log in
> and see if the system already has that derivation running, and if not
> copy over the closure and install it as the system state.  This could
> be entirely declarative.

Right, since these are pet servers, the user already knows all the
relevant information about them.

> I think initially we can ignore entirely the issue of provisioning an
> initial guix image.  We can also ignore things like getting the right
> public keys and configuring the initial authority of how you can be
> authorized to copy over that data.

Yes, the pet server use-cases assume the systems have been setup
manually.  Automated provisioning will be relevant for cloud
deployments.

> > * Managing a large number of virtual machines whose names don't matter
> > that you update in-place (proto-cloud scale)
> > * Managing a large number of virtual machines whose names don't matter
> > that are replaced when there is an update (cloud scale)
> > * Managing 1 or more clusters of physical machines (datacenter scale)
> > * Managing 1 or more clusters of physical machines and virtual
> > machines ("corporation with a datacenter that is moving some stuff to
> > the cloud" scale)
>
> For all these, I think there can be one or more abstraction layers that
> build on top of the same declarative primitives described above.  And
> this may be where state comes in; since you don't really know what the
> identifiers are ahead of time, but you do need to keep track of them,
> yes we do want to serialize the state somewhere.

I don't think this tool should ever store state.  For example, if you
have a bunch of EC2 instances, you can use the AWS API to query for
the instances you want to update.  No need for us to worry about state
and deal with the inevitable synchronization issues.

> > There are, of course, more scenarios to consider (haven't even touched
> > upon things like a Kubernetes cluster), but this is enough to
> > illustrate the point that is a great diversity in setups.  How many
> > machines are there? Are the bare metal, virtual machines, or a mix of
> > both? In the case of virtual machines, are updates applied in an
> > immutable fashion or not?  If immutable, which technique (blue-green,
> > rolling release, etc.)?  It makes my head spin to think about all the
> > use-cases.
>
> ... I don't know anything about the things you said above so I guess
> that would be nice to go over in person :)
>
> I think maybe my ignorance is helpful for optimism in this case!
>
> > So... let's start small. Can we write a tool that handles in-place
> > updates to machines (physical or virtual) whose name and IP address we
> > know well (our special pet servers) without precluding the possibility
> > of scaling up to more sophisticated architectures?  This would address
> > the "home" and "blog" scale items above, which is probably what most
> > of the people actually using Guix today would want.  I got stuck
> > trying to do in-place updates to remote machines years ago, but that
> > was before Ludo made it easy to connect to remote systems.
>
> Yes!
>
> Also I think guile-ssh might make things a bit easier than they were in
> the past, too.
>
> Let's do this!

Sounds good!

- Dave