Re: Packaging rust crates without compiling dependencies again for every dependent (wasting build time)

[Efraim Flashner]

On Sat, Apr 30, 2022 at 11:29:25AM +0200, Maxime Devos wrote:
> Hi various distributions packaging rust things (^),
> 
> Currently, most (all?) distributions seem to package ‘source-only
> crates’, which when ‘compiled’, just copy some source code to a
> location.  This works, but the downside is that if both a package X and
> a package Y depends on rust-Z, then rust-Z will effectively be compiled
> twice: when building X, but also when building Y.
> 
> This seems a waste of compilation time to me, so I've been writing a
> build tool ‘antioxidant’ (*) for the distribution Guix that avoids
> double compilation, at
> <https://notabug.org/maximed/cargoless-rust-experiments>).
> 
> To avoid the double compilation things, antioxidant calls the
> underlying rustc by itself instead of cargo, and it reads and
> interprets Cargo.toml by itself instead of letting cargo do that.
> 
> Currently, it's rather Guix-specific, but in principle it should be
> possible to turn it in some binary 'antioxidant' that could be used on
> other distributions as well if someone is interested in that, e.g.
> something like
> 
> $ # (not implemented yet!)
> $ antioxidant configure --prefix=$HOME/extra-software -Lcrate=/usr/lib/crates 
> -L/usr/lib -Lcrate=$HOME/extra-software/lib/crates --features=x,y,z
> $ antioxidant make
> $ antioxidant make check
> $ antioxidant make install DESTDIR=... # this does not install source code, 
> only the compiled crates
> 
> (*) A ‘build system’, in Guix terminology.
> 
> Currently, it can succesfully compile 'hexyl', 'sniffglue' and all
> their dependencies.  I'm working on addressing limitations that
> prevent antioxidant from compiling other binaries (currently trying
> to make 'agate' and all its dependencies compile).
> 
> Greetings,
> Maxime.
> 
> (^) If you are reading this from guix-devel@: some of the distributions I
> intend to contact have a no-crossposting policy, so I'm first sending this to
> guix-devel@, then sending some mails to individual distributions with a link
> to this e-mail.

This is really really cool! Thanks for sharing it.

I noticed that there are a number of crates that you remove
unconditionally, would it help if we patched them out of the crates that
are in Guix?

I'm glad you found a use for (target-64bit?), it seemed too useful to
not have in Guix :)

It took me less than 3 minutes to download and build all the sources for
the three packages! It looks to be faster than building the packages
straight, and with the benefit of incremental compilation.


-- 
Efraim Flashner   <efraim@flashner.co.il>   אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

signature.asc
Description: PGP signature