Re: Boostrap tar cannot exec xz

[Carlos Sánchez de La Lama]

Hi Ludo,

> So apparently, tar 1.27 did not execute xz via /bin/sh, but current tar
> does not either:
>

It does not when extracting, but seems to do when compressing (which is
what causes my bootstrapping problem). Your experiment yields similar
results here:

--8<---------------cut here---------------start------------->8---
$ ./pre-inst-env guix build bootstrap-binaries
/gnu/store/2g5cgcv8fsmaps528jn6np241mkyyidm-bootstrap-binaries-0
$ /gnu/store/2g5cgcv8fsmaps528jn6np241mkyyidm-bootstrap-binaries-0/bin/tar 
--version
tar (GNU tar) 1.29
Copyright (C) 2015 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Written by John Gilmore and Jay Fenlason.
$ strace -e execve -f 
/gnu/store/2g5cgcv8fsmaps528jn6np241mkyyidm-bootstrap-binaries-0/bin/tar xf 
~/foo.tar.xz 
execve("/gnu/store/2g5cgcv8fsmaps528jn6np241mkyyidm-bootstrap-binaries-0/bin/tar",
 ["/gnu/store/2g5cgcv8fsmaps528jn6n"..., "xf", "/home/csanchez/foo.tar.xz"], 
[/* 25 vars */]) = 0
/gnu/store/2g5cgcv8fsmaps528jn6np241mkyyidm-bootstrap-binaries-0/bin/tar: This 
does not look like a tar archive
Process 1608 attached
[pid  1608] execve("/home/csanchez/bin/xz", ["xz", "-d"], [/* 25 vars */]) = -1 
ENOENT (No such file or directory)
[pid  1608] execve("/home/csanchez/bin/xz", ["xz", "-d"], [/* 25 vars */]) = -1 
ENOENT (No such file or directory)
[pid  1608] execve("/home/csanchez/bin/xz", ["xz", "-d"], [/* 25 vars */]) = -1 
ENOENT (No such file or directory)
[pid  1608] execve("/bin/xz", ["xz", "-d"], [/* 25 vars */]) = -1 ENOENT (No 
such file or directory)
[pid  1608] execve("/sbin/xz", ["xz", "-d"], [/* 25 vars */]) = -1 ENOENT (No 
such file or directory)
[pid  1608] execve("/usr/bin/xz", ["xz", "-d"], [/* 25 vars */]) = 0
xz: (stdin): File format not recognized
[pid  1608] +++ exited with 1 +++
--- SIGCHLD {si_signo=SIGCHLD, si_code=CLD_EXITED, si_pid=1608, si_uid=1000, 
si_status=1, si_utime=0, si_stime=0} ---
/gnu/store/2g5cgcv8fsmaps528jn6np241mkyyidm-bootstrap-binaries-0/bin/tar: Child 
returned status 1
/gnu/store/2g5cgcv8fsmaps528jn6np241mkyyidm-bootstrap-binaries-0/bin/tar: Error 
is not recoverable: exiting now
+++ exited with 2 +++
--8<---------------cut here---------------end--------------->8---

However, when *compressing*, it needs /bin/sh.

--8<---------------cut here---------------start------------->8---
$ touch ~/foo
$ strace -f -e execve 
/gnu/store/2g5cgcv8fsmaps528jn6np241mkyyidm-bootstrap-binaries-0/bin/tar cvfa 
~/foo.tar.xz ~/foo
execve("/gnu/store/2g5cgcv8fsmaps528jn6np241mkyyidm-bootstrap-binaries-0/bin/tar",
 ["/gnu/store/2g5cgcv8fsmaps528jn6n"..., "cvfa", "/home/csanchez/foo.tar.xz", 
"/home/csanchez/foo"], [/* 25 vars */]) = 0
/gnu/store/2g5cgcv8fsmaps528jn6np241mkyyidm-bootstrap-binaries-0/bin/tar: 
Removing leading `/' from member names
/home/csanchez/foo
Process 1615 attached
[pid  1615] 
execve("/gnu/store/eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee-bash-4.3.42/bin/sh", 
["/gnu/store/eeeeeeeeeeeeeeeeeeeee"..., "-c", "xz"], [/* 25 vars */]) = -1 
ENOENT (No such file or directory)
tar (child): xz: Cannot exec: No such file or directory
tar (child): Error is not recoverable: exiting now
[pid  1615] +++ exited with 2 +++
--- SIGCHLD {si_signo=SIGCHLD, si_code=CLD_EXITED, si_pid=1615, si_uid=1000, 
si_status=2, si_utime=0, si_stime=0} ---
/gnu/store/2g5cgcv8fsmaps528jn6np241mkyyidm-bootstrap-binaries-0/bin/tar: Child 
returned status 2
/gnu/store/2g5cgcv8fsmaps528jn6np241mkyyidm-bootstrap-binaries-0/bin/tar: Error 
is not recoverable: exiting now
+++ exited with 2 +++
--8<---------------cut here---------------end--------------->8---

This is what makes <hash>-make-4.2.tar.xz-builder to fail, stopping the
bootstrap.

>> Another option that just came to my mind is replacing all the nuked
>> store referenced in the static binaries with references to the unpacked
>> store path after extraction. As the nuked references can be easily
>> identified by the "eeeeee..." fake path, this should be doable.
>
> That’s not doable because we don’t know the hash in advance.

When creating the tarball, we don't know the hash it will have when
extracted, so the references are nuked. But when creating the package
from the tarball, in the bootstraping machine (my linux-powerpc in this
case), couldn't we "un-nuke" the references, making them point to the
store path we just created?

Thanks!

Carlos