bug#74217: Bootstrapping Zig with no Binary Blobs

[Hilton Chain]

Hello everyone,

With a new forced push (and new rebuilds, sorry), wip-zig-bootstrap is mostly
ready (for me) now.  Please take a look at the first few commits, as I'm
changing Zig's behavior there, here are some additional notes:

  To Efraim: Can adding pkg-config to native-inputs avoid the ncdu snippet?

I'm building this branch on my personal Cuirass instance[1][2], for x86_64-linux
and aarch64-linux (qemu-binfmt), in previous revisions I have indentified
reproduciblity issue, and aarch64 builds timed-out.  I haven't investigated them
yet.

Then for what's the RUNPATH issue I have mentioned in commits:
+ For current zig@0.10 on Guix master: glibc is missing from RUNPATH, which
fails the validate-runpath check.
+ For zig@0.11, some other inputs are missing, making the binary failing to run 
on Guix[3].
+ dan also mentioned privately to me that they needed to add paths from
LIBRARY_PATH to RUNPATH for their own projects, so programs built by Zig is also
affected.

I think this is due to Zig's implemention of its own linking logic, which
bypasses our ld-wrapper.

I'm not going to implement ld-wrapper within Zig. :) So my proposed workaround
in wip-zig-bootstrap is to patch the handling logic added for Guix:

(In lib/std/zig/system/NativePaths.zig)
--8<---------------cut here---------------start------------->8---
         // Distros like guix don't use FHS, so they rely on environment
         // variables to search for headers and libraries.
         // We use os.getenv here since this part won't be executed on
         // windows, to get rid of unnecessary error handling.
-        if (std.posix.getenv("C_INCLUDE_PATH")) |c_include_path| {
+        if (std.posix.getenv("CROSS_C_INCLUDE_PATH") orelse 
std.posix.getenv("C_INCLUDE_PATH")) |c_include_path| {
             var it = mem.tokenizeScalar(u8, c_include_path, ':');
             while (it.next()) |dir| {
                 try self.addIncludeDir(dir);
             }
         }

-        if (std.posix.getenv("CPLUS_INCLUDE_PATH")) |cplus_include_path| {
+        if (std.posix.getenv("CROSS_CPLUS_INCLUDE_PATH") orelse 
std.posix.getenv("CPLUS_INCLUDE_PATH")) |cplus_include_path| {
             var it = mem.tokenizeScalar(u8, cplus_include_path, ':');
             while (it.next()) |dir| {
                 try self.addIncludeDir(dir);
             }
         }

-        if (std.posix.getenv("LIBRARY_PATH")) |library_path| {
+        if (std.posix.getenv("CROSS_LIBRARY_PATH") orelse 
std.posix.getenv("LIBRARY_PATH")) |library_path| {
             var it = mem.tokenizeScalar(u8, library_path, ':');
             while (it.next()) |dir| {
                 try self.addLibDir(dir);
+                try self.addRPath(dir);
             }
         }
     }
--8<---------------cut here---------------end--------------->8---

Adding directories from CROSS_LIBRARY_PATH or LIBRARY_PATH to RUNPATH, "CROSS_"
part is for our cross toolchain, I haven't tested it yet.

I think this behavior change is reasonable since the search path
(CROSS_)?LIBRARY_PATH is only automatically set by our compilers.

I added this change to 0.9 as well to make all Zigs behave consistently.  I also
used shrink-runpath phase from meson-build-system in Zig and zig-build-system.

I want to move shrink-runpath to (guix build utils) and export it too, so that
it can be used easier.  But I'm not sure if this change will trigger rebuilds of
other packages, so I didn't do it.

Thanks to Guile, for builds not managed by guix-daemon, something like the
following script can be used, we can ship a program-file if we agree on this
workaround.
--8<---------------cut here---------------start------------->8---
(use-modules (guix build meson-build-system))

(define shrink-runpath
  (assoc-ref %standard-phases 'shrink-runpath))

(define (main directories)
  (for-each (lambda (dir)
              (false-if-exception
               (shrink-runpath
                #:elf-directories '(".")
                #:outputs `(("out" . ,dir)))))
            directories))

(main (cdr (command-line)))
--8<---------------cut here---------------end--------------->8---
Usage: guile <file-with-above-content> DIRECTORY...


Thanks
---
[1]: https://ci.boiledscript.com/jobset/guix-zig
[2]: https://substitute.boiledscript.com, if you want to challenge it.
[3]: https://github.com/ziglang/zig/issues/18434