Preservation of Guix Report 2021-11-30

[Timothy Sample]

Hi Guix!

Here’s a new version of the Preservation of Guix Report:

    <https://ngyro.com/pog-reports/2021-11-30/>

I actually made one a month ago but my message about it never made it to
the list somehow.  The most important part of that message was to
highlight how well we are doing for Git sources.

Here’s what I wrote:

>  This version has a breakdown by different origin types.  The good
>  news is that Git origins are doing very well.  We’ve confirmed that
>  97.2% of the 9,272 Git origins that we’re tracking are in the SWH
>  archive.  Most of the progress there is due to zimoun wading through
>  the missing packages and telling SWH to store them – thanks, zimoun!

That’s still basically true this month, but we have a few more missing
Git sources.  Actually, we are starting to lose sources!  If you look at
the graph of commits, you can see a sharp increase in missing sources
for recent commits.  It looks like a problem on the SWH side.  Visiting
[1] and selecting “Show all visits”, you can see that the nixguix loader
has been having trouble loading our “sources.json” recently.

[1] 
<https://archive.softwareheritage.org/browse/origin/visits/?origin_url=https://guix.gnu.org/sources.json>

I will try and get in touch with SWH about this.  While it’s troubling,
it certainly is a good confirmation that doing some basic monitoring is
important!

That’s the bad news.  The good news is I’ve added support for XZ to
Disarchive (to be officially released in Disarchive soon).  That means
that we have information about 4K more sources.  We now know the status
of 80% of our sources.  Unfortunately, 40% of the XZ sources are
missing!  Most of them are old, as can be seen in this (secret) graph:

    <https://ngyro.com/pog-reports/2021-11-30/tar-xz-rel-hist.svg>

(The filename format is “{tar-gz,tar-xz,git}-{rel,abs}-hist.svg” if you
want to see all the secret graphs.)

Lastly, if you scroll to the bottom of the report and select “View
Schema”, I’ve added some example queries that generate lists of
interesting sources.  For example – if you’re so inclined – you could
look at the 128 “unknown”, non-recursive Git sources that we should know
about and figure out what’s going on.  ;)


-- Tim