Progress on 'guix deploy'

[Jakob L. Kreuze]

Hello, Guix!

Apart from a few patches and my introductory email about a month ago,
I've been pretty silent. I feel it's time to finally break that silence,
let people know where progress has been made, and request some feedback
on the code I've written so far.

As a quick refresher, my GSoC project this summer is 'guix deploy', a
deployment automation tool for GuixSD that's been discussed more
thoroughly in [1] and [2]. Development has taken place on my personal
branch of Guix, specifically the 'wip-deploy' branch [3], and is
represented by three new Scheme source files:

- 'gnu/machine.scm', which provides an abstraction for /something/ that
  can be deployed to in a heterogeneous deployment. Currently, the only
  concrete implementation of this is the simple case of in-place updates
  to machines running SSH whose names and IP addresses we know.
- 'gnu/tests/machine.scm', which implements some tests for
  'gnu/machine.scm'. This is where I'm most interested in receiving
  feedback. More on that later.
- 'guix/scripts/deploy.scm', which implements the rudimentary
  command-line interface for 'guix deploy'.

The command-line interface hasn't really been fleshed out yet, but if
you'd like to play around with it, it takes a "deployment" file as a
parameter, which is a Scheme file looking something like the following:

#+BEGIN_SRC scheme
(use-modules ...)

(define %system
  (operating-system
   (host-name "gnu-deployed")
   (timezone "Etc/UTC")
   (bootloader (bootloader-configuration
                (bootloader grub-bootloader)
                (target "/dev/sda")
                (terminal-outputs '(console))))
   (file-systems (cons (file-system
                        (mount-point "/")
                        (device "/dev/vda1")
                        (type "ext4"))
                       %base-file-systems))
   (services
    (append (list (service dhcp-client-service-type)
                  (service openssh-service-type
                           (openssh-configuration
                            (permit-root-login #t)
                            (allow-empty-passwords? #t))))
            %base-services))))

(list (make <sshable-machine>
        #:host-name "localhost"
        #:ssh-port 5556
        #:system %system))
#+END_SRC

Basically, you attach an 'operating-system' declaration to a 'machine'.
In this case, 'sshable-machine' is the specific type of 'machine' that
we're deploying to (one that's running an SSH daemon and has a known IP
+ port + hostname).

I've found that the GuixSD QEMU images work well for playing around with
this, provided that you add the SSH service to the system configuration
and start it. In the case of this deployment file, I had a GuixSD guest
with port 22 forwarded to my host's port 5556. You'll also need to set
up some sort of public key auth in your SSH config. The current code
isn't capable of handling other forms of SSH authentication.

In terms of implementation, GOOPS feels like a bit of an unusual choice
when juxtaposed with the rest of the Guix codebase, but I've come to
really enjoy it. I'll roll with it for now, since I think it will make
it easier to flesh out the vocabulary for specifying deployments.

The implementation of '<sshable-machine>' is doing what
'switch-to-system' and 'install-bootloader' in 'guix/scripts/system.scm'
do, but in terms of data that can be sent with 'remote-eval'. I imagine
the code will make more sense if you read both simultaneously.

Okay, on to the test suite.

My understanding of the system test suite (tests run with 'check-system'
as opposed to those run with 'check') is that the meat of the test code
exists in a G-Expression and should _not_ be interacting with the store
of the machine running the test suite (i.e. that's the reason we're
using marionettes in the first place). 'gnu/tests/install.scm' seems to
be somewhat of an exception, and because the code in '(gnu machine)'
depends heavily on having access to a store, I've tried to extend what's
done in 'gnu/tests/install.scm' so that my tests have access to store
while instrumenting the marionettes.

To be specific, the chicken and egg scenario I'm working around is that
the SSH daemon on the marionette needs to be running in order for
'deploy-os' to work, but I can't call 'deploy-os' from the test
G-Expression because the store wouldn't be accessible then.

My gut is telling me that this is absolutely the wrong way to go about
it, though. 'call-with-marionette' is one of a couple red flags making
me feel that way -- I don't think marionettes were meant to be started
outside the context of a derivation...

If anyone has suggestions on how I might go about properly testing this
code, I would appreciate it very much.

Another point about the test suite: the 'ssh-deploy-os' test fails, but
it's a reproducible version of the issue outlined in [2]. I'd like to
conscript some help from those more familiar with guile-ssh before
breaking out the ol' RFCs myself.

...

So, if anyone's in the mood to peek at what's been written so far and
give feedback, that'd make my day. Doesn't have to be feedback related
to the test code, either. Any sort of comment, be it regarding the code
style or perhaps even some suggestions on improving the interface, would
be appreciated.

I'm going to continue to spend time on the "internals" of 'guix deploy'
this coming week, incorporating any feedback I receive and ensuring that
I have a framework to build upon when I extend this to more complicated
scenarios like deploying to a VPS provider. After that, I'll tackle
fleshing out the way deployments are specified (the file part. I'm
holding off on the command-line tool part for right now).

Signing off for now. Huge thanks to everyone for the warm welcome I
received following my introduction email.

Regards,
Jakob

[1]: https://lists.gnu.org/archive/html/guix-devel/2019-02/msg00145.html
[2]: https://lists.gnu.org/archive/html/guix-devel/2019-03/msg00114.html
[3]: https://git.sr.ht/~jakob/guix/tree/wip-deploy

signature.asc
Description: PGP signature