Re: Guix Docker image inflation

[zimoun]

Dear Stephen,

On Sun, 31 May 2020 at 21:43, Stephen Scheck <singularsyntax@gmail.com> wrote:
> On Sun, May 31, 2020 at 2:51 PM zimoun <zimon.toutoune@gmail.com> wrote:
>>
>> Maybe the explosion of size would be slower.  If you do, please report
>> here the number after say 12 generations; I am really interesting. ;-)
>
> Now I'm confused - in your reply to Vincent, it seemed that there were still 
> problems
> with the GC removing dead store items even after you did an export/re-import 
> with the
> entire image on a single Docker layer? Or did I misread it?

The export/import trick cut by half the size of "your guix-bootstrap"
image.  So even I am not convince that it will fix the issue, I think
that my proposal is the correct thing to do to delete dead items in
the store.  Basically, after the pull, you need to delete all the
other generations of /root/.config/guix/current (expected one) by
"guix pull -d", then to delete all the generations of
/root/.guix-profile with "guix package -d" and last garbage collect.
For sure, it will not delete the items coming from previous layer but
it will delete all the dead items of the current session.  And "docker
export | docker import" could remove other items -- even if in the
case of "install/remove hello" it is not work cleanly, some items are
deleted.

Well, it is just to be complete with your approach.


>> All the question seems to be:
>>  - what is the purpose of such Docker image?  Which usage?
>>  - what infrastructure do you have at hand to build the Docker images?
>
> Well, Guix is interesting, and there aren't ready-made containers for it out 
> there like there are for
> Ubuntu, Fedora, etc. if you have a need to do some task in that kind of 
> environment, or just to play
> around, or see how the system is evolving. Also, I have been playing around 
> with Guile lately and
> I thought Guix might be a better fit for that kind of work than other 
> environments where Guile is
> largely neglected (Guix is *written* in Guile, after all). And I happened to 
> be learning GitLab CI/CD
> around the same time, and it seemed like a good opportunity to experiment 
> with both at once,
> so I thought, why not? :-) Which infrastructure - well, GitLab CI/CD, with 
> fixed compute limits :-)

Yeah, "ready-made container" could be really cool!  AFAIK, no one took
the time to implement and document.  There are various attempts but
not always reported on help-guix or guix-devel.  Well, the answer of
these 2 questions implies different strategies.

For example, I am running Guix on the top of Debian so I basically use
only the package manager.  And I use this infrastructure to produce
Docker images containing apps running "guix pack -f docker -m
manifest.scm".  Because I am interested in Reproducible Science, I
also use "guix time-machine -C channel.scm -- pack -f docker -m
manifest.scm".  However, the Docker images contains only applications
(R or Python with bunch of packages) and the "user" cannot use these
images to extend them running "guix install foo"; because I want to
track reproducibility so the only way is to go by 'manifest.scm' and
'channel.scm' files.

Another example is the Dockerfile way.  Based on any image (Alpine or
Debian), I build an image containing the Guix package manager --
roughly speaking as you are doing with your image 'guix-bootstrap'.
Then I use this image in 2 different ways: with a Dockerfile or
directly.  In both cases, it always starts by "guix pull.  And I never
chain the images -- I mean only 3 "layers" at maximum: 0-debian,
1-guix-fresh and 2-guix-lastest.  Well, I have never run "guix gc"
inside a Docker image.


Last, I have never played with "guix system docker-image".  But in the
context of GitlabCI, what I would try should be something like:

    CONTAINER=`docker run --detach --privileged $OLD`
    docker exec $CONTAINER guix pull
    docker exec $CONTAINER guix system docker-image --root=/image.tar stuff.scm
    docker cp $CONTAINER:$IMG $NEW

with maybe instead of "guix pull" this bazooka:

    docker exec $CONTAINER guix install git
    docker exec $CONTAINER git clone
    docker exec $CONTAINER guix environment guix
    docker exec $CONTAINER ./bootstrap && ./configure --localstatedir=/var/
    docker exec $CONTAINER make -j
    docker exec $CONTAINER ./pre-inst-env guix system docker-image


Well, use "guix system docker-image" inside a Docker image already
containing Guix; this avoid the layer issue, isn't it?


But as I said elsewhere, I am not really familiar with Docker so my
words are probable irrelevant.

All the best,
simon