Patchwork + automated checking and testing of patches

[Christopher Baines]

Hey,

For a while, I've been thinking about ways in which contributing code
changes to Guix can be better, in particular, how things like
automatically checking and testing patches might be beneficial.

Some tooling for doing checks already exists, like guix lint, but you
can't run that on a patch, you need some way of getting from the text
sent to the mailing list to some state in a git repository.

I haven't used Patchwork [1], but it has been tried previously with Guix
[2][3]. It also seems to be a step forward in terms of taking what's
sent to the mailing list and organising it.

1: http://jk.ozlabs.org/projects/patchwork/
2: https://lists.gnu.org/archive/html/guix-devel/2016-05/msg00056.html
3: https://patchwork.sourceware.org/project/guix/list/

I've now written a very rough package and service for Patchwork [4], and
managed to setup a instance here [5]. With the help of an email account
subscribed to both guix-patches and guix-commits, getmail and a couple
of scripts, it should also collect new patches sent to guix-patches, and
mark those that have been merged to the master branch as "Accepted" [6].

4: https://debbugs.gnu.org/cgi/bugreport.cgi?bug=33185
5: https://patchwork.cbaines.net/
6: https://patchwork.cbaines.net/project/guix-patches/list/?state=3

What I'm hoping now to do, is to try and use Patchwork to coordinate
automated checking and testing of patch series. I haven't thought much
about this yet, but Patchwork does have something for recording
"checks", demonstrated here [7]. There has also been a talk at FOSDEM
about this kind of thing [8] as well as a blog post with more detail
[9].

7: https://patches.dpdk.org/patch/43055/
8: https://archive.fosdem.org/2017/schedule/event/patchwork_jenkins/
9: https://that.guru/blog/patchwork-and-ci-in-a-tree/

I don't intend to do anything with Jenkins, as I think that wouldn't be
maintainable, but I think setting up some system to check some of the
following would be beneficial:

 - If a patch series applies to master
 - If ./configure and make run successfully
 - If building a guix package with the patch applied works
 - If running guix lint for all the new/changed packages works
 - If building all the new/changed packages works
 - If running the tests and system tests works

I think there are also some review activities that are better done at
scale. One example I can think of is comparing the contents of source
tarballs and store outputs for new packages to those of existing
packages. If there is some overlap, it may suggest that there is some
bundled code/data that could be removed.

Let me know if you have any thoughts,

Chris

signature.asc
Description: PGP signature