Re: unexpected reproducibility of reproducible blog post?

[Leo Prikler]

Hi zimoun,

Am Montag, den 27.04.2020, 12:05 +0200 schrieb zimoun:
> Hi Leo,
> 
> Thank you for testing.
> 
> 
> On Mon, 27 Apr 2020 at 00:53, Leo Prikler <
> address@hidden> wrote:
> 
> > yours: /gnu/store/klisfr3a4wxb9dc5sgibb45kky72kg65-docker-pack.tar
> > mine:  /gnu/store/klisfr3a4wxb9dc5sgibb45kky72kg65-docker-pack.tar
> 
> Nice!
> 
> What is your "guix describe"?
I used the same time-machine as you did, so I don't think it matters,
but I'm currently at 1e700c656a7a25ff2eb27fa552bc213cc50efb2a.  The
result is the same for 86081d9d7f88a7faee6fd14e8a085cb95ac1e36a (a
"random" commit in January), as long as you remember to actually use
the time-machine to jump back to the commit mentioned in the blog post.

> > I don't know, what configuration exactly went into the blog post,
> > but I
> > assume, it is not the same as for the time-machine experiments
> > before.
> > Since the prefix `guix time-machine --channels=guix-version-for-
> > reproduction.txt --` appears to be missing from the command, that
> > hash
> > is therefore probably not indicative of anything.
> 
> I do not know. That's why I am asking. :-)
> Because when reading the blog post, I naively assume that all had
> been
> run with the same version of Guix and the post mentions only one
> commit. Well, if it is not the case, it should be mentioned in the
> blog post because it is currently misleading, IMO.
I agree.  There should probably be an addendum correcting the
information.  Perhaps adding some up-to-date hashes would probably not
be a bad idea either.

> > I think the larger problem here is that, while Guix itself is
> > reproducible, Guix + org-mode (specifically the latter) is not.
> 
> Why?
There are too many ways to bork org-mode -- you yourself specifically
list one of them later.  I don't think org-mode not being reproducible
is a big secret.

> > Particularly, looking at the source[1,2], it appears as if all code
> > blocks were evaluated once, but evaluating them again in a new
> > environment would bring different results.
> 
> Do you mean evaluate twice in a row leads to different results?
> By results, I mean items in '/gnu/store'.
> Because, yes the org-babel cache should not be reproducible. But that
> another story and should not impact the result of a source block.
That by itself is no biggie, but it becomes particularly bad when you
throw in partial evaluation.  If you don't evaluate all the code blocks
once, your results may get stale and then you'd export those stale
results.  Throw in some command that hasn't been evaluated yet and
evaluate it on export and you get yourself a recipe for deluxe bogus.

> My point is:
>  - only one Guix commit is provided by the post, so it seems
> legitimate to assume this commit had been used for all the post
>  - using this commit leads to different item in the store
This assumption may appear intuitive, but by reproducing the time-
machine on many Guix systems and verifying that it is indeed
reproducible (albeit with a different hash), we can invalidate it.

> The question is why?
>  - another commit had been used. Which one? Could be mentioned in the
> post?
>  - or there is something unexpected and let inspect what.
I assume it was accidental.  Had it been known earlier, that a
different commit was used, the blog post would have been updated
between Jan 14th (last update in git) and Jan 24th (official post),
would it not?

All the best,
Leo