Guix Day: Notes from the CI session

[Ludovic Courtès]

Hello Guix!

Attached are the notes from the “continuous integration” session held a
few hours ago.

Thanks,
Ludo’.

# Continous integration

## Latest changes in Cuirass

  - switch from sqlite to PostgreSQL
  - stop using the built-in offload mechanism
    - take inspiration from Hydra: offload straight from Cuirass to _workers_
    - head node talks to workers over ZMQ
    - head node feches builds from workers upon build completion, triggers 
"substitute baking" (for `guix publish`)
  - new [worker monitoring page](https://ci.guix.gnu.org/workers)
    - shows what each worker is doing
    - soon: will be able to view CPU/storage usage
  - build info now shows a _weather_ icon showing the build status evolution
    - per-specification RSS feeds allow users to monitor, say, `guix-master`
    - maybe soon: user accounts on the web interface
  - new _build priorities_, per spec (`guix-master`, `core-updates`, etc.)
  - `timeout` and `max-silent-time` properties of packages are now honored
  - integration with `guix pull`: `channel-with-available-substitutes`
  - many features in the works (see Mathieu's Org-mode file)
  
## Open issue: new machines

  - fast ARM servers available
  - criteria for hardware?
    - must run free system (stock Guix System)
  - hosting?
    - the MDC (in Berlin) wouldn't host Guix-specific non-x86 servers
    - could rent space (currently renting space for bayfront.guix in Bordeaux, 
France), but it's relatively costly
    - workstations could be in people's homes
      - https://store.avantek.co.uk/ampere-altra-64bit-arm-workstation.html
        - (benchmark info: 
https://www.phoronix.com/scan.php?page=article&item=ampere-altra-q80 - compete 
with AMD Epyc 7nm 2nd Gen)
      - 
    - small boards are unreliable or too slow
    - looking for volunteers to help
      - pick hardware we could buy
      - get in touch on `guix-devel` or `guix-sysadmin` (private) mailing lists

## Open issue: branching strategy

  - currently: building all of `master` + the "core" of `core-updates`
  - schedule
    - currently ad-hoc: volunteers get to choose when to freeze/merge
  - actions
    - pushes to `core-updates` should cancel all pending builds on that branch 
(see Guix Build Coordinator)
    - have a dashboard showing active branches, statuses (wild-west, frozen), 
and merge deadlines
      - could be implemented in Cuirass
      - specs could have extra properties (such as "merge deadline") that 
Cuirass would display
      - or it could use status info from the Data Service, as in 
https://data.guix-patches.cbaines.net/
    - naming convention: `core-updates-frozen` vs `core-updates`, etc.
    
 ## Open issue: missing CI features
  - be able to restart builds
  - detect build return codes: failed because of dependencies, failed because 
of timeout ...
  - short-term actions:
    - allow for more than two builds per worker on `berlin`
    - set a hard limit on the number of emulated builds by workers on `berlin`