bug#73602: bug#73601: failed to compute the derivation for Guix

[Fabrice Tudoret]

Hi Simon,

Thanks again for your involvement.

I did my best to fullfill the tests you suggest. I put the output in your text.

Essentially the output are the same for the root and the users, so the situation does not seem clearer to me, but I’m sure it will be different for you.

Regards

Fabrice T

Le 11/10/2024 à 18:30, Simon Tournier a écrit :

> Hi Fabrice,
>
> On Fri, 11 Oct 2024 at 15:13, Fabrice Tudoret <fabrice.tudoret@univ-rennes1.fr> wrote:
>
> > GUIX seem's to work fine with the root account and the local users but
> > not with ldap users.
> >       
> Ah, that doesn’t ring a bell but maybe it’s related.  I don’t know.
>
>
> > 1-
> >       
> Just to be sure and since you have reinstalled, what is your Guix
> revision?
>     
[root@cluster24 ~]# guix --version
guix (GNU Guix) 7888351b9edd7b0199a973c75bc1c35897d9d7ef
Copyright (C) 2024 the Guix authors
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

> On a side note, since it seems fine for the root account and here you
> run it as root, I guess all is fine. :-)  Although I don’t have the
> same… 🤔
>
> > [root@cluster24 ~]# cat /gnu/store/jc3vgcsplqsim3na80b0n2iilna5j6gx-Python-3.5.9.tar.xz.drv  | sed 's/)/)\n/g'
> > Derive([("out","/gnu/store/cv4h89n30myf3nhjqnnahlbij2gaw21z-Python-3.5.9.tar.xz","","")
> > ],[("/gnu/store/21c7pjahkh20mmzq2ivki57zwwvp6nwn-bootstrap-binaries-0.drv",["out"])
> > ,("/gnu/store/5gf7f8awndhnf2gn2mzbfbqr3ix9aj80-module-import-compiled.drv",["out"])
> > ,("/gnu/store/g08l2msvnivyi6x5nw52ak8n17sw9lzr-guile-bootstrap-2.0.drv",["out"])
> > ,("/gnu/store/lb5b7svdmfj1ijnzrripsjcv0bhqzpwb-Python-3.5.9.tar.xz.drv",["out"])
> > ],["/gnu/store/1s8jdafkyhz0p81l0j37yih9gbrb5gix-module-import","/gnu/store/h58cvdcdak4d87lw0fkvmkhan95ssljx-Python-3.5.9.tar.xz-builder"],"x86_64-linux","/gnu/store/lgi9x15a0w35mcpd7g1kb9274r6wy4pv-guile-bootstrap-2.0/bin/guile",["--no-auto-compile","-L","/gnu/store/1s8jdafkyhz0p81l0j37yih9gbrb5gix-module-import","-C","/gnu/store/gz5rcilhcsc5amgxcgyxvn0s5px8sg80-module-import-compiled","/gnu/store/h58cvdcdak4d87lw0fkvmkhan95ssljx-Python-3.5.9.tar.xz-builder"],[("guix properties","((type . origin)
> >  (patches . 0)
> > )
> > ")
> > ,("out","/gnu/store/cv4h89n30myf3nhjqnnahlbij2gaw21z-Python-3.5.9.tar.xz")
> > ])
> >
> >
> > 2 -The script builder
> > /gnu/store/ykqckrxcmifvxz0nb58lv2drgd14l377-Python-3.5.9.tar.xz-builder
> > is well present in /gnu/store.
> >       
> Ouf. :-)
>
>
> Well, since it works as expected when run as root (#) and it fails when
> run as a regular ($), I propose to diff various files in order to spot
> what could be wrong.
>
> Some details about some internals – well my understanding and I’m
> perhaps missing important points –, then maybe they will explain the
> logic behind the exploration. :-)
>
> Roughly speaking, the items in the store look like:
>
>     /gnu/store/xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx-foobar-1.2.3
>
> where ’foobar-1.2.3’ is a “label” corresponding to the package and
> ’xxx…’ is some hash.  This hash is the core of the content-addressed;
> the one that allow the substitution, i.e., download the artefacts.
>
> Basically, this hash is computed by hashing the inputs and the script
> builders.  Therefore somehow it builds a chain and the roots are named
> fixed-outputs.  Fixed-outputs are items for which we known beforehand
> the resulting hash.  Else we cannot know the hash beforehand because
> it’s hard to know beforehand the checksum of the artefact since the
> artefact is the result of the build process (compilation, etc.).
>
> In other words, the expectation is: the same inputs and the same builder
> script returns the same store item.  And the derivation captures that.
> Well, for sure the content of the store item on two machines is the same
> only if the process is fully deterministic; another story. ;-)
>
> All that to say: if we scrutinize the derivations and the builder
> scripts, then we will spot what introduces a difference.
>
> Aside, please note that two different derivations might produce the same
> store item, see for example [1].
>
> Let’s go! :-)
>
>
> a) Both root and regular must use the exact same Guix revision.
>
>   # As root
>   # readlink -f (type -P guix)
>   # guix describe
>
>     
> [root@cluster24 ~]# readlink -f guix
> /root/guix
> [root@cluster24 ~]# type -P guix
> /root/.config/guix/current/bin/guix
> [root@cluster24 ~]# guix describe
> Generation 3 Oct 14 2024 08:46:12 (current)
> guix 7888351
> repository URL: https://git.savannah.gnu.org/git/guix.git
> branch: master
> commit: 7888351b9edd7b0199a973c75bc1c35897d9d7ef
>     
>   $ As regular user
>   $ readlink -f (type -P guix)
>   $ guix describe
>     
> [fmenna@cluster24 ~]$ readlink -f guix
> /NAS/home/fmenna/guix
> [fmenna@cluster24 ~]$ type -P guix
> /usr/local/bin/guix
> [fmenna@cluster24 ~]$ guix describe
> guix 7888351
> repository URL: https://git.savannah.gnu.org/git/guix.git
> branch: master
> commit: 7888351b9edd7b0199a973c75bc1c35897d9d7ef
>     
> b) If that’s not the case, it isn’t an issue for the following but it’s
>    better to know. :-)
>

     It's seem ok.

> c) Display the both derivations, as root and as regular user
>
>   # guix build -e '(@@ (gnu packages commencement) python-boot0)' -S -d
>     
> [root@cluster24 ~]# guix build -e '(@@ (gnu packages commencement) python-boot0)' -S -d
> /gnu/store/mbnrrqjizy3dvma9i3l8g5yvzi2p04ys-Python-3.5.9.tar.xz.drv
>     
>   $ guix build -e '(@@ (gnu packages commencement) python-boot0)' -S -d
>     
> [fmenna@cluster24 ~]$  guix build -e '(@@ (gnu packages commencement) python-boot0)' -S -d
> user with UID 255759 not found
> /gnu/store/mbnrrqjizy3dvma9i3l8g5yvzi2p04ys-Python-3.5.9.tar.xz.drv
>

I hope the message "user with UID 255759 not found" has nothing to do with the issue.

> The expected situation is: as root,
>
>   # guix build -e '(@@ (gnu packages commencement) python-boot0)' -S
>
> it just downloads stuff but as regular user:
>
>   $ guix build -e '(@@ (gnu packages commencement) python-boot0)' -S
>
> then it builds stuff and one build fails.
>
> Correct?
>

There is no indication that it builds or fails.

> d) Display the content of the both derivations, as root and and regular
>    user.
>
>   drv=$(guix build -e '(@@ (gnu packages commencement) python-boot0)' -d -S)
>   cat $drv | sed 's/),/\n),/g'
>

[root@cluster24 ~]# drv=$(guix build -e '(@@ (gnu packages commencement) python-boot0)' -d -S)
  cat $drv | sed 's/),/\n),/g'
Derive([("out","/gnu/store/jl82wnklrfhjgnml3s3w0l4m4mjwpazb-Python-3.5.9.tar.xz","","")],[("/gnu/store/38951a2gcwb4jyrwlzg634jfwkdih7m2-bootstrap-binaries-0.drv",["out"]
),("/gnu/store/ff107pg13nkarix7k9hgxxwx9cnn4602-module-import-compiled.drv",["out"]
),("/gnu/store/g08l2msvnivyi6x5nw52ak8n17sw9lzr-guile-bootstrap-2.0.drv",["out"]
),("/gnu/store/lb5b7svdmfj1ijnzrripsjcv0bhqzpwb-Python-3.5.9.tar.xz.drv",["out"])],["/gnu/store/jsxgc979x79h81kzqz9n6cpf5pk4z262-module-import","/gnu/store/ykqckrxcmifvxz0nb58lv2drgd14l377-Python-3.5.9.tar.xz-builder"],"x86_64-linux","/gnu/store/lgi9x15a0w35mcpd7g1kb9274r6wy4pv-guile-bootstrap-2.0/bin/guile",["--no-auto-compile","-L","/gnu/store/jsxgc979x79h81kzqz9n6cpf5pk4z262-module-import","-C","/gnu/store/j2r95mqfir1q6hsavm0r22k8z49nbvk1-module-import-compiled","/gnu/store/ykqckrxcmifvxz0nb58lv2drgd14l377-Python-3.5.9.tar.xz-builder"],[("guix properties","((type . origin) (patches . 0))"
),("out","/gnu/store/jl82wnklrfhjgnml3s3w0l4m4mjwpazb-Python-3.5.9.tar.xz")])
[fmenna@cluster24 ~]$  drv=$(guix build -e '(@@ (gnu packages commencement) python-boot0)' -d -S)
  cat $drv | sed 's/),/\n),/g'
user with UID 255759 not found
Derive([("out","/gnu/store/jl82wnklrfhjgnml3s3w0l4m4mjwpazb-Python-3.5.9.tar.xz","","")],[("/gnu/store/38951a2gcwb4jyrwlzg634jfwkdih7m2-bootstrap-binaries-0.drv",["out"]
),("/gnu/store/ff107pg13nkarix7k9hgxxwx9cnn4602-module-import-compiled.drv",["out"]
),("/gnu/store/g08l2msvnivyi6x5nw52ak8n17sw9lzr-guile-bootstrap-2.0.drv",["out"]
),("/gnu/store/lb5b7svdmfj1ijnzrripsjcv0bhqzpwb-Python-3.5.9.tar.xz.drv",["out"])],["/gnu/store/jsxgc979x79h81kzqz9n6cpf5pk4z262-module-import","/gnu/store/ykqckrxcmifvxz0nb58lv2drgd14l377-Python-3.5.9.tar.xz-builder"],"x86_64-linux","/gnu/store/lgi9x15a0w35mcpd7g1kb9274r6wy4pv-guile-bootstrap-2.0/bin/guile",["--no-auto-compile","-L","/gnu/store/jsxgc979x79h81kzqz9n6cpf5pk4z262-module-import","-C","/gnu/store/j2r95mqfir1q6hsavm0r22k8z49nbvk1-module-import-compiled","/gnu/store/ykqckrxcmifvxz0nb58lv2drgd14l377-Python-3.5.9.tar.xz-builder"],[("guix properties","((type . origin) (patches . 0))"
),("out","/gnu/store/jl82wnklrfhjgnml3s3w0l4m4mjwpazb-Python-3.5.9.tar.xz")])

There is no difference in those output between root and regular user.

> Now we can try to find what can be wrong. :-)
>
>
> Roughly, the inputs are one fixed-output and these three others:
>
>     /gnu/store/g08l2msvnivyi6x5nw52ak8n17sw9lzr-guile-bootstrap-2.0.drv
>     /gnu/store/38951a2gcwb4jyrwlzg634jfwkdih7m2-bootstrap-binaries-0.drv
>     /gnu/store/ff107pg13nkarix7k9hgxxwx9cnn4602-module-import-compiled.drv
>
> Well, please note that the hash of these derivations might vary but not
> their output.  For instance,
>
>     # sed 's/]/\n]/g' \
>        /gnu/store/38951a2gcwb4jyrwlzg634jfwkdih7m2-bootstrap-binaries-0.drv \   
>       | head -1
>     Derive([("out","/gnu/store/az8fi8xzgvkqw9aynxks98h29afal6i2-bootstrap-binaries-0","","")
>
> Here the hash ’3895…’ might be different but not ’az8fi8…’.  The same
> derivation produces the same store item though.
>
> Ok, let consider each.  I know that piping with “sed” is not handy and
> an extension [2] ]is still pending… Anyway, please pipe the derivations
> in order to extract the relevant information.
>     
[root@cluster24 ~]# sed 's/]/\n]/g' \                                                    
      /gnu/store/38951a2gcwb4jyrwlzg634jfwkdih7m2-bootstrap-binaries-0.drv | head -1
Derive([("out","/gnu/store/az8fi8xzgvkqw9aynxks98h29afal6i2-bootstrap-binaries-0","","")
[fmenna@cluster24 ~]$ sed 's/]/\n]/g' \
>  /gnu/store/38951a2gcwb4jyrwlzg634jfwkdih7m2-bootstrap-binaries-0.drv | head -1
Derive([("out","/gnu/store/az8fi8xzgvkqw9aynxks98h29afal6i2-bootstrap-binaries-0","","")

> 1. /gnu/store/…-guile-bootstrap: it only depends on fixed-outputs so the
> only source of variation comes from the script builder: in my case,
>
>     /gnu/store/nmxl6qyj16bv4rx4irhg23r66gn752kd-build-bootstrap-guile.sh
>

I have the same:

[root@cluster24 ~]# find /gnu/store -name "*build-bootstrap-guile.sh"
/gnu/store/nmxl6qyj16bv4rx4irhg23r66gn752kd-build-bootstrap-guile.sh

> I expect you have the same.  As root, you have because it’s the same
> derivation, namely
> /gnu/store/g08l2msvnivyi6x5nw52ak8n17sw9lzr-guile-bootstrap-2.0.drv.
>
> What about the regular user?
>

The same too.

> 2. /gnu/store/…-bootstrap-binaries: the inputs reads,
>
>     /gnu/store/ff107pg13nkarix7k9hgxxwx9cnn4602-module-import-compiled.drv
>     /gnu/store/g08l2msvnivyi6x5nw52ak8n17sw9lzr-guile-bootstrap-2.0.drv
>     /gnu/store/nf9h1l2apgjq1gjdv1ynpgqrmaj1bawb-static-binaries.tar.xz.drv
>     /gnu/store/sik0rp63q7hz4wry27v3ynljdmqvg194-xz.drv
>     /gnu/store/zn60yim3km5qx8d0vzvf9izfxzfdy4x0-tar.drv
>
> Assuming #1 (identical guile-bootstrap) and the only non fixed-output is
> module-import-compiled.  Or the script builder.
>
> Do you have the same script builder, namely
>
>   /gnu/store/kv2vjnp7f75wnq8rp579ihz96knqp0sl-bootstrap-binaries-0-builder
>
> ?  I guess you have the same.  Therefore, I guess the issue is from
> module-import-compiled.
>

I've the same:

[root@cluster24 ~]# find /gnu/store -name "*bootstrap-binaries-0-builder"
/gnu/store/kv2vjnp7f75wnq8rp579ihz96knqp0sl-bootstrap-binaries-0-builder

> 3. /gnu/store/…-module-import-compiled: On my machine, it reads:
>
> --8<---------------cut here---------------start------------->8---
> Derive
> ([("out","/gnu/store/j2r95mqfir1q6hsavm0r22k8z49nbvk1-module-import-compiled","","")]
>  ,[("/gnu/store/g08l2msvnivyi6x5nw52ak8n17sw9lzr-guile-bootstrap-2.0.drv",["out"])]
>  ,["/gnu/store/jsxgc979x79h81kzqz9n6cpf5pk4z262-module-import","/gnu/store/l36kgiw25r2a4vllwad18jg8jnkvmzda-build-utils.scm","/gnu/store/rn7b0dq6iqfmmqyqzamix2mjmfygn31n-compile-modules"]
>  ,"x86_64-linux","/gnu/store/lgi9x15a0w35mcpd7g1kb9274r6wy4pv-guile-bootstrap-2.0/bin/guile",["--no-auto-compile","/gnu/store/rn7b0dq6iqfmmqyqzamix2mjmfygn31n-compile-modules"]
>  ,[("GUILE_WARN_DEPRECATED","no")
>    ,("extensions","")
>    ,("module count","1")
>    ,("modules","/gnu/store/jsxgc979x79h81kzqz9n6cpf5pk4z262-module-import")
>    ,("optimization level","1")
>    ,("out","/gnu/store/j2r95mqfir1q6hsavm0r22k8z49nbvk1-module-import-compiled")
>    ,("preferLocalBuild","1")])
> --8<---------------cut here---------------end--------------->8---
>
> I guess you also have the same build-utils.scm.  Don’t you?
>

I've the exactly the same content in the file /gnu/store/ff107pg13nkarix7k9hgxxwx9cnn4602-module-import-compiled.drv

[root@cluster24 ~]# cat /gnu/store/ff107pg13nkarix7k9hgxxwx9cnn4602-module-import-compiled.drv
Derive([("out","/gnu/store/j2r95mqfir1q6hsavm0r22k8z49nbvk1-module-import-compiled","","")]
,[("/gnu/store/g08l2msvnivyi6x5nw52ak8n17sw9lzr-guile-bootstrap-2.0.drv",["out"])]
,["/gnu/store/jsxgc979x79h81kzqz9n6cpf5pk4z262-module-import","/gnu/store/l36kgiw25r2a4vllwad18jg8jnkvmzda-build-utils.scm","/gnu/store/rn7b0dq6iqfmmqyqzamix2mjmfygn31n-compile-modules"]
,"x86_64-linux","/gnu/store/lgi9x15a0w35mcpd7g1kb9274r6wy4pv-guile-bootstrap-2.0/bin/guile",["--no-auto-compile","/gnu/store/rn7b0dq6iqfmmqyqzamix2mjmfygn31n-compile-modules"]
,[("GUILE_WARN_DEPRECATED","no")
,("extensions","")
,("module count","1")
,("modules","/gnu/store/jsxgc979x79h81kzqz9n6cpf5pk4z262-module-import")
,("optimization level","1"),("out","/gnu/store/j2r95mqfir1q6hsavm0r22k8z49nbvk1-module-import-compiled")
,("preferLocalBuild","1")])

About the build-utils.scm, they are also identical:

/root/.cache/guix/checkouts/pjmkglp4t7znuugeurpurzikxq3tnlaywmisyr27shj7apsnalwq/tests/build-utils.scm
/NAS/home/fmenna/.cache/guix/checkouts/pjmkglp4t7znuugeurpurzikxq3tnlaywmisyr27shj7apsnalwq/tests/build-utils.scm

> Therefore, I guess the difference is either an environment variable (the
> list starting with GUILE_WARN_DEPRECATED) or the file
> /gnu/store/…-qcompile-modules.  Hum, it would be weird that it comes
> from compile-modules…
>

> Ah my train is at destination.  To be continued… Let me know!
>
> Cheers,
> simon
>
>
> 1: Re: Derivations differ between computers?
> zimoun <zimon.toutoune@gmail.com>
> Fri, 26 Nov 2021 01:49:00 +0100
> id:86lf1bspvn.fsf@gmail.com
> https://lists.gnu.org/archive/html/guix-devel/2021-11
> https://yhetil.org/guix/86lf1bspvn.fsf@gmail.cmo
>
> 2: Guix extension to display derivation (and rewrite fixed-output)
> Simon Tournier <zimon.toutoune@gmail.com>
> Fri, 12 Apr 2024 20:28:11 +0200
> id:87ttk6phac.fsf@gmail.com
> https://lists.gnu.org/archive/html/guix-devel/2024-04
> https://yhetil.org/guix/87ttk6phac.fsf@gmail.com
>

smime.p7s
Description: Signature cryptographique S/MIME