Re: Self-contained Guix tarball

[Ludovic Courtès]

Pjotr Prins <address@hidden> skribis:

> When you administrate a large amount of servers things can go wrong
> due to system failures, failed backup recoveries, hacking attempts and
> adminstrators trying to be clever ;). Murphy's law dictates that the
> store and the sqlitedb meta information will go out of sync. For
> production setups it is necessary to be able to recover from backups,
> but as an intermediate recovery step it would be really nice if Guix
> could recover its meta information from an existing store - assuming
> only the DB is corrupt (google for sqlite corrupted databases). It is
> especially nice when your backups are out of sync too.

The important thing is that currently, the DB is authoritative.  So it
cannot be corrupt (that would be equivalent to having lost /gnu/store
altogether), and thus it cannot be repaired.

What *can* be repaired is the store: for instance, if a store item is
tampered with.  The daemon has code to do it, but the Guix client tools
do not expose it yet.

> guix archive looks good, but for speedy deployment it can happen an 
> adminstrator would have a simple use case of:
>
> - Copying 2 stores to 1 machine
> - Rebuild database
>
> to do it quick and dirty. Since all software packages are isolated
> this would make a really good use case even if (with some trouble) you
> could use guix archive for that. I prefer quick and dirty.
>
> I am not pushing for this functionality directly, but I would
> certainly like to have it when I need it :)

I don’t think it could work the way you envision it.  What kind of
deployment do you have in mind?  For whole system deployment, one can
obviously use ‘guix system’.

>> I suspect this would make GC inefficient (lots of disk seeks to
>> determine references/referrers compared to queries of the SQLite
>> database.)
>
> Yes, Nix switched to using sqlitedb because of the GC.

I think it’s been there “forever” (at least since I started contributing
in 2008.)

> It is also useful to search current versions of installed packages
> quickly. Even so, I think it should be viewed as an index. The state
> of the machine is what is *sitting* in the store. That would be the
> correct design.
>
> Meta information can go out of sync. Therefore we should not assume
> they are in sync.

Again, the store can go “out of sync,” but the DB itself is
authoritative currently.

And it’s important that it be this way.  One example is that build
processes can write their outputs to the store; if the build process
fails, there are still those files in the store, but the DB won’t have
recorded them as valid, so they can be swept on the next GC.

>> Another (opposite :-)) option is to make /gnu/store a read-only bind
>> mount on GuixSD.  Commit 3392ce5 does that.  This will prevent
>> accidental modifications of the store.
>
> That is a good solution for end-users. Not for administrators. So
> adminstrators will circumvent it.

Well, administrators won’t be able to circumvent it accidentally, at
least.

I understand this framework really constrains what sysadmins can do, and
in particular prevents them from doing “quick-and-dirty hacks.”  I think
we should strive to find the UIs that allow for quick hacks while not
compromising the store’s integrity.

WDYT?

Thanks,
Ludo’.