[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: “guix gc”, auto gcroots, cluster deployments
From: |
Roel Janssen |
Subject: |
Re: “guix gc”, auto gcroots, cluster deployments |
Date: |
Mon, 10 May 2021 17:45:30 +0200 |
User-agent: |
Evolution 3.40.1 (3.40.1-1.fc34) |
On Mon, 2021-05-10 at 13:59 +0200, Ricardo Wurmus wrote:
>
> Hi Roel,
>
> thanks for your feedback!
>
> > Would it be possible to add an option to retrospectively apply
> > this
> > transformation? Maybe that could work somewhat like this:
> >
> > $ ls -lha
> > ... /home/me/projects/mrg1_chipseq/.guix-profile-1-link ->
> > /gnu/store/ap0vrfxjdj57iqdapg8q83l4f7aylqzm-profile
>
> This wouldn’t work, because we can’t read
> /home/me/projects/mrg1_chipseq/.guix-profile-1-link centrally. In
> this particular case only the user “me” could resolve the link and
> thus migrate the link. (I would do this semi-manually by
> impersonating the users to read their links.)
>
Indeed we cannot resolve /home/me/projects/mrg1_chipseq/.guix-profile-
1-link as another user than "me". So "me" would have to run the
hypothetical "guix gc --synchronize-profiles-to-gcroots".
>From my point of view, that would be fine. We can simply ask the users
on the cluster to execute that command once.
This could work if we have a mechanism to determine how complete the
garbage-collection picture for "root" is, and only continue GCing when
the picture is complete. A starting point for this is
$localstatedir/gcroots/auto which provides an overview of all profiles
that have been created. Perhaps we can reuse the filename of the
symlink. So in the example given in the previous e-mail we could add
another link like so, but in a different directory:
8ypp8dmwnydgbsgjcms2wyb32mng0wri -> /gnu/store/...-profile
For which this already existed in $localstatedir/gcroots/auto:
8ypp8dmwnydgbsgjcms2wyb32mng0wri ->
/home/me/projects/mrg1_chipseq/.guix-profile-1-link
And then a one-time transition can be made by looking up
8ypp8dmwnydgbsgjcms2wyb32mng0wri in both places.
Maybe I'm making things needlessly complex, but I would really like a
solution that can be applied to an existing cluster setup. :)
So, your initial proposal sounds good to me already. How it could be
made more useful for the retrospective matching of unreachable symlinks
to /gnu/store paths can be considered a separate issue to solve.
> ~ ~ ~
>
> Another related problem I found is that the names of the links in
> unreadable space may have names that only make sense on the system
> where they were created. For example, on the server “beast” we
> may mount the cluster home directory as “/clusterhome/me”, whereas
> on cluster nodes it would be mounted as “/home/me”. When I have
> Guix record a gcroot while working on “beast” I would get a link
> that is no longer valid when I work on the cluster.
>
Yes, I agree that this is another reason to implement your suggested
change. :)
Kind regards,
Roel Janssen
- “guix gc”, auto gcroots, cluster deployments, Ricardo Wurmus, 2021/05/10
- Re: “guix gc”, auto gcroots, cluster deployments, Sébastien Lerique, 2021/05/10
- Re: “guix gc”, auto gcroots, cluster deployments, Ludovic Courtès, 2021/05/11