Re: ZFS on Guix

[raid5atemyhomework]

Greetings guix-developers and other people trying to use Guix,

The patchset currently dying on issues.guix.gnu.org would provide a nice simple 
single-step way to enable *very basic* ZFS support on your Guix system.  Until 
it gets merged, however, you can still enable *very very basic* ZFS support on 
your Guix system by following the below minimal guide.

First, select a Linux kernel that the latest ZFS package on Guix supports.  
Current on Guix is ZFS 2.0.1.  That supports up to Linux 5.10.  So I suggest 
using a `linux-libre-5.10` kernel.  Don't go with the default `linux-libre` or 
`linux-libre-lts` since those could be updated to past what the ZFS on Guix 
supports, be explicit with what kernel version you have.  Just remember to 
update to a later LTS version if ZFS on Guix ever gets updated to a later 
version.

Then, you need to create a ZFS package, by adding some declarations before the 
`operating-system` form:

```scheme
(use-modules #;...
             (guix packages))
(use-package-modules #;... linux file-systems)

(define my-kernel linux-libre-5.10)
(define my-zfs
  (package
    (inherit zfs)
    (arguments (cons* #:linux my-kernel (package-arguments zfs)))))
```

Then in the `operating-system` form, you need to add the following bits:

* Add ZFS to the kernel-loadable modules so that the installed kernel knows how 
to read and write ZFS disks.
* Add ZFS to the system profile packages so that you can easily manage ZFS 
disks.

So you have to modify a number of places:

```scheme
(operating-system
  (kernel my-kernel)
  (kernel-loadable-modules (list (list my-zfs "module")))
  #;...
  (packages
    (append (map specification->package (list "nss-certs"))
            (list my-zfs)
            %base-packages))
  #;...)
```

With the above you get a ***ridiculously minimal*** ZFS support.

* You have to `sudo modprobe zfs` explicitly at startup.
* You have to `sudo zpool import -a` explicitly at startup.

To do the above automatically at startup you need to add a Shepherd service to 
do the `zpool import`, and add a `kernel-module-loader` extension.  This 
requires a bit more code to be added to your `configuration.scm`.

Here's what I got in my `configuration.scm`:

```scheme
(define zfs-shepherd-services
  (let ((zpool            (file-append my-zfs "/sbin/zpool"))
        (zfs              (file-append my-zfs "/sbin/zfs"))
        (scheme-modules   `((srfi srfi-1)
                            (srfi srfi-34)
                            (srfi srfi-35)
                            (rnrs io ports)
                            ,@%default-modules)))
    (define zfs-scan
      (shepherd-service
        (provision '(zfs-scan))
        (documentation "Scans for ZFS pools.")
        (requirement '(kernel-module-loader udev))
        (modules scheme-modules)
        (start #~(lambda _
                   (invoke/quiet #$zpool "import" "-a" "-N")))
        (stop #~(const #f))))
    (define zfs-automount
      (shepherd-service
        (provision '(zfs-automount))
        (documentation "Automounts ZFS data sets.")
        (requirement '(zfs-scan))
        (modules scheme-modules)
        (start #~(lambda _
                   (with-output-to-port
                     (current-error-port)
                     (lambda ()
                       (invoke #$zfs "mount" "-a" "-l")))))
        (stop #~(lambda _
                  (chdir "/")
                  (invoke/quiet #$zfs "unmount" "-a" "-f")
                  #f))))
    (list zfs-scan
          zfs-automount)))
```

Then, add some `simple-service`s to your `operating-system`:

```scheme
(use-modules
  #;...
  (gnu services))
(use-service-modules linux shepherd #;...)

#;...

(operating-system
  #;...
  (services
    (append
      (list
        #;...
        (simple-service 'zfs-loader
                        kernel-module-loader-service-type
                        '("zfs"))
        (simple-service 'zfs-shepherd-services
                        shepherd-root-service-type
                        zfs-shepherd-services)
        (simple-service 'zfs-shepherd-services-user-processes
                        user-processes-service-type
                        '(zfs-automount))
        #;...)
      #;...
      %desktop-services))
  #;...)
```

The above lets you mount ZFS pools automatically at startup.  Encrypted pools 
with `keylocation=prompt` will prompt at the console on bootup.

Caveats:

* You can't have a `/home` on ZFS mount.  ZFS has to be mounted before 
`file-systems` target starts, otherwise Guix will fill up the root-mounr's 
`/home` directory and ZFS will refuse to mount over that.  No `/` or `/boot` on 
ZFS either.  Probably no good way to put `/gnu/store` on ZFS either, because 
Guix inherently assumes it's on the `/` mount.
* The above setup does not maintain a `/etc/zfs/zpool.cache` file, because I'm 
not really certain whether it's kosher in Guix to have a file maintained by ZFS 
in the `/etc` dir hierarchy.  This has a number of consequences:
  * `zdb` doesn't work because it looks for `/etc/zfs/zpool.cache`.  Good luck 
trying to figure out why your pool is slow.
  * You can't practically have more than a few dozen disks in your system, 
because the above uses `zpool import -a` which will cause ZFS to scan all the 
disks at bootup which would probably be slow if you have lots of disks.
* You can't practically use `zvol`s to back other filesystems in such a way 
that you can put them in a `file-system` declaration.  Though why use any 
file-system other than ZFS amirite.  You can still use `zvol`s to e.g. back a 
virtual machine that you launch manually when the system boot is finished, or 
from a Shepherd service that explicitly lists `user-processes` as a 
requirement.  Though hmmm the above doesn't use `zvol_wait` anywhere... sigh.
* There's no ZED.  No automatic replacement of failing drives with a hot spare. 
 No monitoring.  You can probably try launching it in its own Shepherd service, 
but you need to figure out how to populate `/etc/zfs/zed/` yourself.  If you 
do, you probably will not be doing it from the `configuration.scm` file meaning 
it'll be hard to replicate the setup elsewhere.


Thanks
raid5atemyhomework