Re: guix package is slow

[Maxim Cournoyer]

Hi!

address@hidden (Ludovic Courtès) writes:

> Hello!
>
> Maxim Cournoyer <address@hidden> skribis:
>
>> address@hidden writes:
>>
>>> Hi
>>>
>>> I would like guix package -A/s to be faster.
>>> There it takes >3s every time I run the command.
>
> It’s at 1.5s for me on an SSD and a hot cache, but I agree that overall
> it’s too slow.
>
>> I agree that Guix is rather slow when comparing it to traditional
>> package managers such as apt; I think the main reason, as you found, is
>> that it has to load all the byte-compiled package definition files.
>
> That, and also it generally has “more work” to do (see ‘apt-get update’
> vs. ‘guix pull’, or even ‘guix package -i emacs’ vs. ‘apt-get install
> emacs’.)  That’s not meant as an excuse, just a clarification of what’s
> going on.

True, especially for guix pull ;)

>>> Would it be possible to populate an index or something when pulling so
>>> that we have the information in a sqlite-db instead of traversing 40
>>> go-files with guile?
>>
>> I remember asking Ludovic in #guix if Guile would be able to compete
>> with database software in terms of performance, and he said there are
>> ways we can make it faster.
>
> I think we could easily optimize package lookups by name.  Instead of
> traversing the whole list of package modules to build up a name/package
> table, we could have a pre-built cache containing that mapping.  That
> way, when typing “guix build emacs”, we could essentially translate that
> to “guix build -e '(@ (gnu packages emacs) emacs)'”, which is a little
> bit faster.
>
> We could arrange for the cache to be enough for operations such as “guix
> package -A”, so that we don’t need to load a single package module.
>
>> I'm not very knowledgeable myself about Guile but if we could make it
>> support static linking, to load only one binary rather than hundreds
>> in the case of Guix, that would be one way to speed things. Next thing
>> might be to implement native compilation.
>
> You might like this:
>
>   https://lists.gnu.org/archive/html/guile-devel/2018-06/msg00026.html

Interesting developments are coming to Guile, it seems :). I'm happy to
see that Andy is still driving Guile forward. Thanks!

>>> On an older guixsd with about 4k packages the delay was noticeably
>>> shorter. 
>>
>> I guess the time it takes grows linearly (I hope!) with the number of
>> packages/modules it has to load. We're now at double that amount of
>> packages (IIRC), so Guix might take double the time it used to.
>
> Yeah, that’s why the cache is probably unavoidable.

Thanks for sharing your insights on the matter!

Maxim