Re: Autodafe is "production" ready

[jbranso]

June 2, 2024 at 5:39 PM, "Eric S. Raymond" <esr@thyrsus.com> wrote:



> 
> jbranso@dismail.de <jbranso@dismail.de>:
> 
> > 
> > Hey Guix!
> > 
> >  
> > 
> >  Have you heard about autodafe? I just did today.
> > 
> >  
> > 
> >  So it looks like autodafe (1) converts autotools into 
> > 
> >  simple makefiles. Quoting from it's hacker's guide (2):  
> > 
> >  "The proximate goal of this project is to eliminate autotools's
> > 
> >  piles of intermediate products and scripts that create a 
> > 
> >  jungle in which exploits like the xz crack of 2024 can hide.
> > 
> >   The wider goal is to abolish the festering complexity sink 
> > 
> >  that autotools recipes have become."  Sounds like a fairly cool
> > 
> >  goal eh?
> > 
> 
> I should emphasize that the project was *directly* motivated by the xz
> 
> Trojan. That's what, in my mind, moved autotools from "chronic
> 
> irritation" to "Someone should kill it with fire, *now*, and I guess
> 
> that will be me". Took me eight weeks of balls-to-the-wall hacking.
> 
> The demand for some solution to supply-chain attacks through autotools
> 
> is urgent. I've been getting a particular earful about it from a
> 
> friend who is on the Gentoo packager lists; they want to be able to
> 
> throw out autotools *yesterday*. I have no doubt similar conversations
> 
> are going on within other packaging groups.
> 
> > 
> > I know that guix has wanted to create a "guile based build 
> > 
> >  system" to replace autotools (autodafe is written in 
> > 
> >  python 3).  It sounds like Eric's work is moving to 
> > 
> >  supplant and/or fix or repace autotools.  Eric is there
> > 
> >  anything Guix can do for you to help you with your goal?
> > 
> >  We do love our guile, so we will probably encourage you
> > 
> >  to use some scheme code.  :)
> > 
> 
> I looked into using Guile embedded in Make as an engine;
> 
> I'm a fan of Scheme myself. There are two reasons I didn't
> 
> go that route:
> 
> 1. Practical: Guile support is normally disablled in the packaged
> 
> versions of gmake on Debian, Ubuntu, Red Hat, etc.
> 
> That means that for my purpose of fielding a prompt replacement for
> 
> autotools, Guile might as well not be there at all. The demand for a
> 
> usable drop-in replacement for autotools won't wait for the number of
> 
> packaging cycles it would take to get all the distributions on board
> 
> with enabling embedded Guile-in-make and then deploying it.
> 
> 2. Architectural: I have a promary goal of making the build
> 
> flow in the replacement for autotools more auditable, with fewer
> 
> places an attacker can hide exploit components.
> 
> As much as I personally like Scheme, it would be a move in the wrong
> 
> direction from an audiability perspective even if it were
> 
> available. Too few people can read Scheme, or want to be able to.
> 
> What Scheme might have done if it were actually available I have
> 
> managed to cover with 661 lines of standalone Python 3. Under
> 
> the requirements, that is in every way a better solution,
> 
> even if not having an excuse to hack some Lisp makes me
> 
> a little wistful.
> 
> > 
> > Richard do you have any advice for Eric in how to "fix or
> > 
> >  replace" autotools?
> > 
> >  
> > 
> >  Ludo, does guix need to do anything to get ready for some
> > 
> >  software wanting to use autodafe  ?  Is there a way we 
> > 
> >  could use autodafe to help us get started on a guile based
> > 
> >  build tool?
> > 
> 
> You'll make a better decision about that if you understand
> 
> what I've actually done.
> 
> There are three tools in the autodafe suite. Two of them are one-time
> 
> transition aids; the third replaces autotools configure and is meant
> 
> to be checked into project repositories and shipped with them.
> 
> deconfig analyzes a code tree, removes guards abound POSIX features,
> 
> and reports a minimal set of guard symbols for non-POSIX
> 
> extensions. On a typical large old project, assuming POSIX reduces the
> 
> complexity of the configuration about an order of magnitude - you go
> 
> from hundreds of goard symbols to dozens.
> 
> Yes, this does mean you risk losing back-portability to big-iron
> 
> Unixes from the last century. One of the design premises of autodafe
> 
> is that in 2024 this is an acceptable trade for a huge
> 
> reduction in the complexity and obscurity of the build recipe.
> 
> makemake takes a Makefuke generated by autotools and tries to rurn it
> 
> into a standalone Makefile that can be read and modifies by humans -
> 
> removing unused boilerplate and rearranging parts so the control
> 
> variables of the build are easy to modify.
> 
> These are two transion aids. They aren't needed onece you have your
> 
> nuild conversion done.
> 
> autodafe configure replaces the autotools generated configure script,
> 
> but is much, *much* smaller and easier to read than those are. It
> 
> reads magic comments in your Makefile (which you add during the
> 
> conversion process - deconfig will typically generates most of them
> 
> for you) to know which testrs to do to discover the capabilities of
> 
> your host ststem. Here's what the example from the documentation
> 
> looks like:
> 
> # CHECK_ENABLE(ipv6)
> 
> # CHECK_DECL(sys/socket.h, netinet/in.h, netinet/tcp.h, TCP_KEEPCNT)
> 
> # CHECK_DECL(sys/socket.h, netinet/in.h, netinet/tcp.h, TCP_KEEPIDLE)
> 
> # CHECK_DECL(sys/socket.h, netinet/in.h, netinet/tcp.h, TCP_KEEPINTVL)
> 
> # CHECK_HAVE(getopt.h, getopt_long)
> 
> # CHECK_HAVE(_GNU_SOURCE, unistd.h, getresuid)
> 
> # CHECK_LIB(m) REQUIRED
> 
> # CHECK_LIB(nsl)
> 
> # CHECK_LIB(resolv)
> 
> # CHECK_LIB(rt)
> 
> # CHECK_HAVE(linux/if/tun.h)
> 
> # CHECK_HAVE(linux/ipv6.h)
> 
> # CHECK_HAVE(_GNU_SOURCE, string.h, memrchr)
> 
> # CHECK_HAVE(_GNU_SOURCE, unistd.h, setresuid)
> 
> # CHECK_SIZEOF(int)
> 
> # CHECK_SIZEOF(int *)
> 
> # CHECK_SIZEOF(long)
> 
> # CHECK_SIZEOF(long long)
> 
> # CHECK_SIZEOF(sys/types.h, off_t)
> 
> # CHECK_SIZEOF(pthread.h, pthread_t)
> 
> # CHECK_SIZEOF(stddef.h, size_t)
> 
> The effect is that the Makefile becomes the single point
> 
> of truth for the build, repl;acing configure.ac.
> 
> The entire transition process is described in detail here:
> 
> https://gitlab.com/esr/autodafe/-/blob/master/configure.adoc?ref_type=heads
> 
> > 
> > Thanks,
> > 
> >  
> > 
> >  Joshua
> > 
> >  
> > 
> >  
> > 
> >  1) https://www.phoronix.com/news/Autodafe-1.0-Released 
> > 
> >  
> > 
> >  2) https://gitlab.com/esr/autodafe/-/blob/master/hacking.adoc
> > 
> 
> -- 
> 
>  <a href="http://www.catb.org/~esr/%22>Eric 
> http://www.catb.org/~esr/%22%3EEric  S. Raymond</a>
>


Eric thanks for the super quick response!  I really enjoyed reading your
previous email.  Thanks for explaining the decisions that you made when
you wrote autodafe.  I suppose autodafe doesn't have a logo yet.  

If there are any good guix artists out there, then you could probably
help Eric by creating a logo for autodafe!  Eric would you mind if I 
took your previous email and turn it into a blog post on
https://gnucode.me  ?

Thanks,

Joshua

P.S.  The Cathedral and the Bazaar was a really fun read for me!