Re: Hurd substitute availability (27.5%) and next steps?

[jbranso]

I'd be happy to reformat this as a guix blog post, unless you'd rather
I not.  

Thanks,

Joshua

March 8, 2021 4:57 PM, "Christopher Baines" <mail@cbaines.net> wrote:

> Hey,
> 
> So I finally got around to trying the Guix Build Coordinator agent on
> the Hurd [1], and builds have been happening on the guix.cbaines.net
> build farm that I've been using to test the Guix Build Coordinator when
> building things for substitutes.
> 
> 1: https://lists.gnu.org/archive/html/guix-devel/2021-02/msg00223.html
> 
> The tooling I've got for looking at how far things have progressed isn't
> great, but either there are blocking builds or some that have got stuck,
> as the agents aren't particularly active currently. Anyway, currently
> for one recent revision (1462a11dbb3d2256c8693e56a583cfd100e27609) I
> tried, guix weather reports 27.5% of substitutes are available. [2]
> 
> → guix weather --system=i586-gnu --substitute-urls=https://guix.cbaines.net 
> computing 10,415 package derivations for i586-gnu...
> looking for 10,811 store items on https://guix.cbaines.net...
> updating substitutes from 'https://guix.cbaines.net'... 100.0%
> https://guix.cbaines.net
> 27.5% substitutes available (2,977 out of 10,811)
> 4,589.5 MiB of nars (compressed)
> 18,975.5 MiB on disk (uncompressed)
> 0.214 seconds per request (2,313.6 seconds in total)
> 4.7 requests per second
> (continuous integration information unavailable)
> 
> The Guix Build Coordinator does have a rudimentary feature to find
> "blocking" builds, and it reports these as the top derivations that are
> failing, and blocking the highest number of builds (in order from top to
> bottom), there are more failures than this, but this is just the top
> few.
> 
> /gnu/store/xrinbhmmsyyxbr755f4b9xiwi1x4lna8-util-linux-2.35.1.drv
> /gnu/store/sgs5vq0y49xhjpc3avi3zpvawzda2j55-tcl-8.6.10.drv
> /gnu/store/nzn4iq4pj65ll0k9c1hkmq83v6czxb3n-zstd-1.4.4.drv
> /gnu/store/cvcra95jcy42s9z99r9mkv6kiqkhr8s3-tcsh-6.22.02.drv
> /gnu/store/qpd0sa86slxz6gb8lzlsybsk8fcd8wzm-valgrind-3.16.1.drv
> /gnu/store/23pw46xhzvnghi6ycpcmj7jz01sz7vaz-apr-1.6.5.drv
> /gnu/store/cq1kpfb8f76rss3kcc5jkaqagcmrf832-ruby-2.6.5.drv
> /gnu/store/jrmm3v34rnsynh59fxjgpi6vkip3slx4-jemalloc-5.2.1.drv
> /gnu/store/lry6i5280aryvmr92jalb5drfd7bj912-libcap-2.31.drv
> /gnu/store/qbf27f3ymxli367nl82cq2z3xh71frm0-elfutils-0.182.drv
> /gnu/store/zfz9hglpn87hwwdf3n3izrrwksb35kl0-kmod-27.drv
> /gnu/store/0lgxyqiaxc5hxyfmkyzzfb2sa5ac9g93-alsa-lib-1.2.4.drv
> /gnu/store/s3zn4wzyi88vyj0cjd8vvpph1ldzg361-libxshmfence-1.3.drv
> /gnu/store/b5awhhqsvgf4c2mxanq2p06nmq1vsgf0-gsl-2.6.drv
> /gnu/store/zqbpy5668wmhpjx6pcdyj1vc6w5iz4n1-linux-pam-1.5.1.drv
> /gnu/store/h0h1yvq4crcbi9rf9cpn95hkdxaigb7h-openblas-0.3.9.drv
> /gnu/store/72b2k49d4kqmnrf0dbjf3bp78nj038d2-jemalloc-4.5.0.drv
> /gnu/store/gvnhx587vsh8l32gq0gfpr8czh2fvzqm-nspr-4.29.drv
> /gnu/store/qfcgw634621gjwswpcq0c8vi5y2rn435-libcap-2.45.drv
> /gnu/store/dzfhypl0vdil3d6kp0w0zcarijbr6shi-flac-1.3.3.drv
> /gnu/store/5cv7sbimd774j6zck18v0pazlh200fb6-libaio-0.3.112.drv
> /gnu/store/0n6fxczd0asf50daznzm4c8m2rw9yk1n-mrustc-0.9.drv
> /gnu/store/3x86mmnf8l2hj1rf74hm8ljbzihj7br0-mtdev-1.1.6.drv
> /gnu/store/l4c3jw3225nwd1q66snhxszgjqa8ir35-libtirpc-1.2.5.drv
> /gnu/store/gcf220qpz1pjk56qp6pxvi302mg7l185-cdparanoia-10.2.drv
> 
> You can find details about packages and builds from this Guix Data
> Service instance [2], the builds can be found on this page for example
> [3].
> 
> 2: https://data.guix-patches.cbaines.net
> 3:
> https://data.guix-patches.cbaines.net/revision/1462a11dbb3d2256c8693e56a583cfd100e27609/builds?build
> server=3&system=i586-gnu&target=none&limit_results=50
> 
> The util-linux and tcl failures are both test related, and I believe
> building without the tests works. With this data, given a package which
> isn't build for the Hurd, say git as an example, it's possible to find
> out which relevant builds are failing [4].
> 
> 4:
> https://data.guix-patches.cbaines.net/build-server/3/build?build_server_build_id=4ebf32a7-5b1e-425f-
> 9e9-cd3933875da3
> 
> So, what are the next steps for Hurd stuff in Guix?
> 
> In terms of getting more packages building, and substitute availability,
> personally I think it would be useful to disable tests for packages for
> i586-gnu if that gets packages building. It's not ideal to not run the
> tests, but it's also difficult to investigate the failures and develop
> patches if you don't have substitutes for the software you need to do
> that (like Git for example).
> 
> More generally, I also know of a few important Hurd related issues:
> 
> guix gc support:
> https://issues.guix.gnu.org/42047
> 
> chroot for builds:
> https://issues.guix.gnu.org/43857
> 
> decisions around what's in the build environment:
> https://guix.gnu.org/en/blog/2020/childhurds-and-substitutes
> 
> I had a go at getting a childhurd with a swap partition, but didn't get
> that far:
> 
> https://issues.guix.gnu.org/46726
> 
> I have reliability issues with the childhurd VMs running Guix Build
> Coordinator agents, often I'll be unable to SSH in, and I'm not quite
> sure how to debug this. It would be nice to run childhurd VMs for the
> guix-patches build coordinator, but ideally they would run reliably and
> wouldn't get stuck.
> 
> I'm quite new to playing around with the Hurd stuff, but I think it's
> quite cool, although I probably need to get back to working more on the
> core Guix Data Service and Guix Build Coordinator features.
> 
> Have I missed something? It would be nice to be able to collect the bugs
> related to the Hurd, I'm not sure how best to do that with debbugs
> though?
> 
> Any thoughts about next steps?
> 
> Thanks,
> 
> Chris