Re: Autodafe is "production" ready

[Eric S. Raymond]

jbranso@dismail.de <jbranso@dismail.de>:
> Hey Guix!
> 
> Have you heard about autodafe?  I just did today.
> 
> So it looks like autodafe (1) converts autotools into 
> simple makefiles. Quoting from it's hacker's guide (2):  
> "The proximate goal of this project is to eliminate autotools's
>  piles of intermediate products and scripts that create a 
> jungle in which exploits like the xz crack of 2024 can hide.
>  The wider goal is to abolish the festering complexity sink 
> that autotools recipes have become."  Sounds like a fairly cool
> goal eh?

I should emphasize that the project was *directly* motivated by the xz
Trojan. That's what, in my mind, moved autotools from "chronic
irritation" to "Someone should kill it with fire, *now*, and I guess
that will be me".  Took me eight weeks of balls-to-the-wall hacking.

The demand for some solution to supply-chain attacks through autotools
is urgent. I've been getting a particular earful about it from a
friend who is on the Gentoo packager lists; they want to be able to
throw out autotools *yesterday*. I have no doubt similar conversations
are going on within other packaging groups.

> I know that guix has wanted to create a "guile based build 
> system" to replace autotools (autodafe is written in 
> python 3).  It sounds like Eric's work is moving to 
> supplant and/or fix or repace autotools.  Eric is there
> anything Guix can do for you to help you with your goal?
> We do love our guile, so we will probably encourage you
> to use some scheme code.  :)

I looked into using Guile embedded in Make as an engine;
I'm a fan of Scheme myself.  There are two reasons I didn't
go that route:

1. Practical: Guile support is normally disablled in the packaged
versions of gmake on Debian, Ubuntu, Red Hat, etc.

That means that for my purpose of fielding a prompt replacement for
autotools, Guile might as well not be there at all.  The demand for a
usable drop-in replacement for autotools won't wait for the number of
packaging cycles it would take to get all the distributions on board
with enabling embedded Guile-in-make and then deploying it.

2. Architectural: I have a promary goal of making the build
flow in the replacement for autotools more auditable, with fewer
places an attacker can hide exploit components.

As much as I personally like Scheme, it would be a move in the wrong
direction from an audiability perspective even if it were
available. Too few people can read Scheme, or want to be able to.

What Scheme might have done if it were actually available I have
managed to cover with 661 lines of standalone Python 3. Under
the requirements, that is in every way a better solution,
even if not having an excuse to hack some Lisp makes me
a little wistful.

> Richard do you have any advice for Eric in how to "fix or
> replace" autotools?
> 
> Ludo, does guix need to do anything to get ready for some
> software wanting to use autodafe  ?  Is there a way we 
> could use autodafe to help us get started on a guile based
> build tool?

You'll make a better decision about that if you understand
what I've actually done.

There are three tools in the autodafe suite.  Two of them are one-time
transition aids; the third replaces autotools configure and is meant
to be checked into project repositories and shipped with them.

deconfig analyzes a code tree, removes guards abound POSIX features,
and reports a minimal set of guard symbols for non-POSIX
extensions. On a typical large old project, assuming POSIX reduces the
complexity of the configuration about an order of magnitude - you go
from hundreds of goard symbols to dozens.

Yes, this does mean you risk losing back-portability to big-iron
Unixes from the last century.  One of the design premises of autodafe
is that in 2024 this is an acceptable trade for a huge
reduction in the complexity and obscurity of the build recipe.

makemake takes a Makefuke generated by autotools and tries to rurn it
into a standalone Makefile that can be read and modifies by humans -
removing unused boilerplate and rearranging parts so the control
variables of the build are easy to modify.

These are two transion aids. They aren't needed onece you have your
nuild conversion done.

autodafe configure replaces the autotools generated configure script,
but is much, *much* smaller and easier to read than those are. It
reads magic comments in your Makefile (which you add during the
conversion process - deconfig will typically generates most of them
for you) to know which testrs to do to discover the capabilities of
your host ststem.  Here's what the example from the documentation
looks like:

# CHECK_ENABLE(ipv6)
# CHECK_DECL(sys/socket.h, netinet/in.h, netinet/tcp.h, TCP_KEEPCNT)
# CHECK_DECL(sys/socket.h, netinet/in.h, netinet/tcp.h, TCP_KEEPIDLE)
# CHECK_DECL(sys/socket.h, netinet/in.h, netinet/tcp.h, TCP_KEEPINTVL)
# CHECK_HAVE(getopt.h, getopt_long)
# CHECK_HAVE(_GNU_SOURCE, unistd.h, getresuid)
# CHECK_LIB(m) REQUIRED
# CHECK_LIB(nsl)
# CHECK_LIB(resolv)
# CHECK_LIB(rt)
# CHECK_HAVE(linux/if/tun.h)
# CHECK_HAVE(linux/ipv6.h)
# CHECK_HAVE(_GNU_SOURCE, string.h, memrchr)
# CHECK_HAVE(_GNU_SOURCE, unistd.h, setresuid)
# CHECK_SIZEOF(int)
# CHECK_SIZEOF(int *)
# CHECK_SIZEOF(long)
# CHECK_SIZEOF(long long)
# CHECK_SIZEOF(sys/types.h, off_t)
# CHECK_SIZEOF(pthread.h, pthread_t)
# CHECK_SIZEOF(stddef.h, size_t)

The effect is that the Makefile becomes the single point
of truth for the build, repl;acing configure.ac.

The entire transition process is described in detail here:

https://gitlab.com/esr/autodafe/-/blob/master/configure.adoc?ref_type=heads

> Thanks,
> 
> Joshua
> 
> 
> 1) https://www.phoronix.com/news/Autodafe-1.0-Released 
> 
> 2) https://gitlab.com/esr/autodafe/-/blob/master/hacking.adoc

-- 
                <a href="http://www.catb.org/~esr/";>Eric S. Raymond</a>