Re: Linux-libre 5.8 and beyond

[Alexandre Oliva]

On Aug 15, 2020, Mark H Weaver <mhw@netris.org> wrote:

> I only checked your claims regarding 5.4, and found that you're mistaken
> about them being updated in 5.4.44.

There was a change to scripts at 5.4.44, just not one you cared about,
because you didn't use the (discontinued) deblob-main script to prepare
a cleaned-up source tarball.

> Moreover, of the 4 deblob updates (.14, .18, .27, and .34) that have
> *actually* been made so far during the 5.4.x series, IIUC only one of
> them declared new blobs to remove, namely the update for 5.4.27.

That's missing the point.  Nearly all of these changes were motivated by
changes reported as suspicious in our verification.  Some turned out to
be false positives, but they might as well have been new blobs.  Any
change has a potential to introduce new blobs, and the fact that our
verification catches suspicious changes that you'd have quietly
published as Free Software is the risk you're passing on to your users
instead of living up to the expectation that you're doing your best to
ensure they're not getting any non-Free Software from you.  The value we
provide, of checking that for every release, you're throwing on the
floor.

Yes, the releases that would *actually* introduce undesirable changes,
vs merely suspicious ones that turn out to be false positives, are a
smaller fraction of the total.  But what you're doing right now is
driving with blinders on because then you can go faster, because history
has shown there's only a 5% or 2% chance of hitting a bus.

> The 5.4.14 update only removed extraneous backslashes in existing
> regexps, changing "\e" to "e" and "\@" to "@".

That was in response to a change in python3.7 (?) regexp engine.
Fortunately, all one got from the extraneous backslashes were warnings.
But it could have been an actual change in output, or a failure to match
a pattern that ought to have been cleaned up, and since you don't
compare with our releases, you could have got non-Free results as much
as from a newly-introduced bit from upstream.

> I don't know whether these extraneous backslashes caused blobs to be
> included in the linux-libre tarballs, but if so, that presumably
> already happened in 5.4.13 and would have happened even if we had used
> your official tarballs, no?

No.  If we'd hit it ourselves, our release engineering procedures would
have caught the unexpected change.  That's why treating our scripts
(rather than our releases) as the ultimate truth, is error prone: the
underlying tools are complex and subject to change and bugs.

If you don't verify that their output isn't garbage (by comparing with
our manually verified releases, or by performing equivalent automated
and manual checks), you may end up shipping that garbage.  Odds are that
you already have.

> The 5.4.18 and 5.4.34 updates only added new 'accept' directives.  I
> guess that means that temporarily omitting these additions wouldn't
> cause new blobs to be included, is that right?

You're probably right for these instances, but it does not necessarily
follow that script changes that only add 'accept' patterns wouldn't get
you in trouble without them.  At times, we've had to add accept
statements to match newly-added occurrences of '.firmware' in such
constructs as:

struct foo var = {
       .whatever = value,
       .firmware = "filename",
       ...
};

These initializers are regarded as suspicious, so they need to be
manually marked as accepted, whether or not the filename turns out to be
a blob name that we clean up.

Without arranging for a newly-introduced '.firmware' initializer to be
accepted, this may end up cleaned up into:

struct foo var = {
       .whatever = value,
       /*(DEBLOBBED)*/ "/*(DEBLOBBED)*/",
       ...
};

which will get you a successful cleaning up session (say, if the
firmware name was already known, in a file that we already cleaned up),
and even a successful compilation, but, depending on the order of the
fields in struct foo, the cleaned-up firmware name may end up used to
initialize the wrong field.



>>> I know this because I always check for updates to the deblob scripts
>>> whenever I update linux-libre in Guix.  In practice, the deblob scripts 
>>> used by
>>> Guix are never more than 1 or 2 micro versions behind the version of
>>> Linux they are applied to.
>> 
>> There have been 61 script updates for the 1274 4.*.*-gnu* and 5.*.*-gnu*
>> stable releases, so Guix has shipped potentially non-FSDG code, that
>> *would* have been flagged by deblob-check on the tarballs, at between 5%
>> and 10% of these releases.  Does that sound like a good standard for a
>> freedom-first distro to aim for?

> If it were true that we've been including blobs in 5-10% of our
> linux-libre releases, I agree that would be a serious problem.

Not what I meant, FWIW.  What I meant was that in 5-10% of the times you
might have *known* you had something wrong in your cleaned up tree if
you'd just run deblob-check on it for one of the automated
verifications.


> I already wrote about 5.4 above.  If we include only the deblob updates
> that added checks for new blobs, it's only happened once in 58 upstream
> updates, i.e. for 1.7% of the updates.

The statistics you're using, counting only the suspicious changes that
were not false positives, is analogous to saying that jaywalking, or
driving across a red light, without even looking, are acceptable as long
as you don't get hit or caught.

Getting lucky 90%, 95% or even 98% of the time doesn't make up for
disregarding the procedures that would have warned you of avoidable
issues, whether or not they turn out to be actual freedom issues.


The other reason you got much lower results than me was that I made room
for your recipe's lagging for up to 2 releases (thus the 5% of stable
releases requiring deblobbing changes turn to 10%), as you'd said, while
you seem to have done yours assuming they'd lag for at most 1.


-- 
Alexandre Oliva, happy hacker
https://FSFLA.org/blogs/lxo/
Free Software Activist
GNU Toolchain Engineer