Re: [PATCH] Help Ruby packages be reproducible

[Ben Woodcroft]

On 31/12/15 03:26, Ludovic Courtès wrote:
> Ben Woodcroft <address@hidden> skribis:
>
> > On 29/12/15 15:46, Ben Woodcroft wrote:
> > > Unfortunately none of these builds are reproducible because rubygems
> > > in Guix generally aren't. For one, this is because .gem files are
> > > archives whose contents are timestamped.
> > I should clarify. What I meant was the cache .gem files
> >
> > /gnu/store/ib83mg5zsyr5x2w0m3i1f84gdvdbp5x9-ruby-ascii85-1.0.2/lib/ruby/gems/2.2.0/cache$
> > tar tvf Ascii85-1.0.2.gem |head
> > -r--r--r-- wheel/wheel     703 2015-12-27 22:44 metadata.gz
> > -r--r--r-- wheel/wheel    7436 2015-12-27 22:44 data.tar.gz
> > -r--r--r-- wheel/wheel     268 2015-12-27 22:44 checksums.yaml.gz
> We should arrange so that gems are created with a fixed timestamp and
> UID/GID, and a well-defined file ordering, as with:
>
>    address@hidden --sort=name --owner=root:0 --group=root:0
>
> We also need to make sure gzip is always run with -n/--no-name.  That
> way, the gz files above will not include an additional timestamp.
>
>  From what I can see in
> <git://git.debian.org/git/reproducible/notes.git>, this is not addressed
> yet in other distros.
Ludo are you suggesting we should abandon the deletion approach?


On 30/12/15 18:26, Ricardo Wurmus wrote:
> Ben Woodcroft <address@hidden> writes:
> > The .gem file stored in GEM_HOME after install is both redundant and an
> > archive that stores timestamped files which makes builds non-deterministic. So
> > delete it after 'gem install'.
> Good idea!  I don’t know if the existence of the cached gem is checked
> for by any Ruby tools (bundler or the like).  Is there some
> documentation about this cache?
I wondered that too, but I built all of the ruby packages again without 
issue and many of them use bundler. It also doesn't seem like a good 
idea for bundler to use cached gems since I would guess that gems that 
are downloaded but fail to install are kept in the cache. I also wasn't 
able to see any mention of the cache in the rubygems API.
> > -    (zero? (apply system* "gem" "install" (first-matching-file "\\.gem$")
> > -                  "--local" "--ignore-dependencies"
> > -                  ;; Executables should go into /bin, not /lib/ruby/gems.
> > -                  "--bindir" (string-append out "/bin")
> > -                  gem-flags))))
> > +    (apply system* "gem" "install" gem-name
> > +           "--local" "--ignore-dependencies"
> > +           ;; Executables should go into /bin, not /lib/ruby/gems.
> > +           "--bindir" (string-append out "/bin")
> > +           gem-flags)
> > +    ;; Remove the cached gem file as this is unnecessary and contains
> > +    ;; timestamped files rendering builds not reproducible.
> > +    (delete-file (string-append gem-home "/cache/" gem-name))
> > +    #t))
> I’d prefer to keep ‘(zero? ...)’ and only delete the file when the
> ‘system*’ call above succeeded.  It would be nice if we could propagate
> any bad return value from ‘system*’ to the end of the procedure.
>
> Maybe something like this:
>
>    (and (zero? (apply system* ...))
>         (begin (delete-file ...) #t))
>
> It’s a bit clunky but the return value would still be #f if ‘system*’
> fails.  What do you think?
I think you are right as usual. Better in attached?

Thanks,
ben

0001-build-ruby-Remove-cached-gem-after-install.patch
Description: Text Data