[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: File search progress: database review and question on triggers
From: |
Mathieu Othacehe |
Subject: |
Re: File search progress: database review and question on triggers |
Date: |
Tue, 11 Aug 2020 11:43:42 +0200 |
User-agent: |
Gnus/5.13 (Gnus v5.13) Emacs/26.3 (gnu/linux) |
Hello Pierre,
Thanks for sharing your progress. A few remarks below.
> 3. Size of the database:
> I've persisted all locally-present store items for my current Guix version
> and it produced a database of 72 MiB. It compresses down to 8 MiB in zstd.
>
> But since we can have multiple Guix versions, this means that the
> packages have one entry per store path, so we might end up with more
> entries than that as the number of Guix generations grows.
I'm not sure we actually need to save the full history. I think we could
just store that the package X produces [Y1, Y2, ...] executable files. Then
on package X update, the executable files list could be refreshed.
> Question: Should we include empty directories in the database? I'm tempted
> to answer no.
I would also say no, and also exclude non-executable files.
> Question: This bounds us to the SQLite syntax for pattern matching. Is it
> a
> problem?
> It seems powerful enough in practice. But maybe we can use regular
> expression in SQLite as well?
>From the UI perspective, we already have "guix search" that expects a
regex. If we were to include a "guix file-search" command, then I think
it would make sense that it uses the same regex syntax.
> Next points I'd like to address:
>
> 6. Automatically persist the database entry when building a package.
> Any idea where I should plug that in?
>
> 7. Have substitute servers distribute database content. When the user
> performs
> a file search, Guix asks the substitute server for a database update. Only
> the diff should be sent over the network, not the whole thing since it
> might
If I understand correctly, you are proposing to create local databases
that would be kept in sync with a master database populated by the CI
server. This seems a bit complex.
What about extending Cuirass database to add the two tables you are
proposing. Then, each time a package is built, if the version is
updated, the "Files" table would be updated.
Then we could add an HTTP interface such as "/search/file?query=libxxx"
to Cuirass, that would directly query the database. In Guix itself, we
could add the counterpart in the (guix ci) module.
WDYT?
Thanks,
Mathieu
- File search progress: database review and question on triggers, Pierre Neidhardt, 2020/08/10
- Re: File search progress: database review and question on triggers,
Mathieu Othacehe <=
- Re: File search progress: database review and question on triggers, Ricardo Wurmus, 2020/08/11
- Re: File search progress: database review and question on triggers, Pierre Neidhardt, 2020/08/11
- Re: File search progress: database review and question on triggers, Pierre Neidhardt, 2020/08/11
- Re: File search progress: database review and question on triggers, Ricardo Wurmus, 2020/08/11
- Re: File search progress: database review and question on triggers, Pierre Neidhardt, 2020/08/12
- Re: File search progress: database review and question on triggers, Julien Lepiller, 2020/08/12
- Re: File search progress: database review and question on triggers, Pierre Neidhardt, 2020/08/12
- Re: File search progress: database review and question on triggers, Julien Lepiller, 2020/08/12