Re: GNU Coding Standards, automake, and the recent xz-utils backdoor

[Jacob Bachmeyer]

Eric Gallager wrote:
> On Sun, Mar 31, 2024 at 3:20 AM Jacob Bachmeyer <jcb62281@gmail.com> wrote:
>   
> > dherring@tentpost.com wrote:
> >     
> > > [...]
> > The issue seems to be releases containing binary data for unit tests,
> > instead of source or scripts to generate that data.  In this case, that
> > binary data was used to smuggle in heavily obfuscated object code.
> >
> > [...]
> >     
>
> Maybe this is something that the GNU project could start making
> stronger recommendations about.
>   

The key issue seems to be generating binary test data during `make` or 
`make check`, using GNU poke, GNU Awk, Perl, Tcl, small C programs, or 
something else, instead of packaging it in the release.  The xz-utils 
backdoor was smuggled into the repository wrapped in compressed test data.

> > > With a reproducible build system, multiple maintainers can "make dist"
> > > and compare the output to cross-check for erroneous / malicious dist
> > > environments.  Multiple signatures should be harder to compromise,
> > > assuming each is independent and generally trustworthy.
> > >       
> > This can only work if a package /has/ multiple active maintainers.
> >     
>
> Well, other people besides the maintainers can also run `make dist`
> and `make distcheck`. My idea was to get end-users in the habit of
> running `make distcheck` themselves before installing stuff. And if
> that's too much to ask of end users, I'd also point out that there are
> multiple kinds of maintainer: besides the upstream maintainer, there
> are also usually separate distro maintainers. Even if there's only 1
> upstream maintainer, as was the case here, I still think that it would
> be good to get distro maintainers in the habit of including `make
> distcheck` as part of their own release process, before they accept
> updates from upstream.
>   

The problem with that is that `make distcheck` only verifies that the 
working tree can produce a reasonable release tarball.  The backdoored 
xz-utils releases *would* *have* *passed* *this* *test* as far as I can 
determine.  It catches errors like omitting files from the lists in 
Makefile.am.  It will *not* catch a modified m4 file or questionable 
test data that has been properly listed as part of the release.

> > > Maybe GNU should establish a cross-verification signing standard and
> > > "dist verification service" that automates this process?  Point it to
> > > a repo and tag, request a signed hash of the dist package...  Then
> > > downstream projects could check package signatures from both the
> > > maintainer and such third-party verifiers to check that nothing was
> > > inserted outside of version control.
> > >       
> > Essentially, this would be an automated release building service:  upon
> > request, make a Git checkout, run autogen.sh or equivalent, make dist,
> > and publish or hash the result.  The problem is that an attacker who
> > manages to gain commit access to a repository may be able to launch
> > attacks on the release building service, since "make dist" can run
> > scripts.  The service could probably mount the working filesystem noexec
> > since preparing source releases should not require running (non-system)
> > binaries and scripts can be run by directly feeding them into their
> > interpreters even if the filesystem is mounted noexec, but this still
> > leaves all available interpreters and system tools potentially available.
> >     
>
> Well, it'd at least make things more difficult for the attacker, even
> if it wouldn't stop them completely.
>   

Actually, no, it would open a *new* target for attackers---the release 
building service itself.  Mounting the scratchpad noexec would help to 
complicate attacks on that service, but right now there is *no* central 
point for an attacker to hit to compromise releases.  If a central 
release building service were set up, it would be a target, and an 
attacker able to arrange a persistent compromise of the service could 
then tamper with later releases as they are built.  This should be 
fairly easy to catch, if an honest maintainer has a secure environment, 
("Why the **** does the central release service tarball not match mine?  
And what the ******** is the extra code in this diff between its tarball 
and mine!?") but there is a risk that, especially for large projects, 
maintainers start relying on the central release service instead of 
building their own tarballs.

The problem here was not a maintainer with a compromised system---it 
seems that "Jia Tan" was a malefactor's sock puppet from the start.


-- Jacob