Re: continuous integrations

[Simon Josefsson]

Bruno Haible <bruno@clisp.org> writes:

>> I think the pattern of having the .gitlab-ci.yml outside of the core
>> Savannah project is a good one for those GNU projects who are not
>> embracing the GitLab platform.  Then GitLab-related stuff stays on the
>> GitLab platform and doesn't invade the core project.
>
> Yes, that's one reason I put the CI outside the main repository. The
> other reasons are:
>   - CIs will come and go over time. Whereas the source code is meant to
>     be stable for > 20 years.
>   - Maintaining CIs is a different business than developing. It can be
>     handled by different persons, with different skills.
>   - I had problems creating a git repository's mirror from Savannah at
>     GitLab. If we can't have a GNU package's mirror at GitLab, and of
>     course don't want to move the main repository away from Savannah,
>     that was the only option.
>   - There is also the possibility of having CIs on other clouds, such as
>     GitHub, Travis, etc. This is simpler if there is no mirroring
>     in-between.

Right.  I also had trouble with Savannah git mirrors in the past, but
for the past year or so it has worked well.  So I like this pattern.

One of the few disadvantages with this approach that I've discovered is
that you don't get tight coupling of ci/cd script and the rest of the
repository.  This means that if you for some reason want to redo the
pipeline on commit X in say 5 years, you may have to find whatever old
commit of the CI pipeline job definition was used at the time and then
set that up to be able to run the pipeline.  If the pipeline definition
can be written to work with both current master git and 5 years old git,
then it will work fine, but it means more work to keep it tested.  I've
found this pattern useful once in a while, but it is not a strong
reason.

>> Then we can apply that group for free CI/CD minutes
>
> What do you mean by that? I've found GitLab's limit of 400 minutes per
> month and top-level group limiting, and see that GitHub does not have such
> a limit.

I have applied for this program for a couple of programs and while it is
a manual process and takes some time, it will give you 50.000 compute
minutes per month:

https://about.gitlab.com/solutions/open-source/join/

By using a single project it would also be possible to purchase compute
minutes in bulk and have them apply to all sub-projects.  I've found
this to be fairly cheap compared to alternative cost of setting up and
maintaining runners on my own hardware.  I've found it to only be cost
effective to setup my own runners for platforms that gitlab doesn't
support natively, such as arm64 or ppc64el.

>> How about using https://gitlab.com/gnulib/ as a playground for these
>> ideas?  Then we can add sub-projects there for pipeline definitions, and
>> Savannah mirrors of other projects too.
>
> On GitLab, the 400 minutes limit is per top-level group. Therefore, it's
> better if, for each GNU package, we have a separate top-level group.

Now I understand why you went through that effort to create new projects!

>> If you can add 'jas' as
>> maintainer of the 'gnulib' group on GitLab
>
> Done.

Thank you.

>> I could add one project to
>> start work on writing re-usable pipeline definitions, and one example
>> project maybe for GNU InetUtils that would use these new re-usable
>> pipeline components to provide a CI/CD pipeline definition file.  I
>> could add some arm64/ppc64el builds of gnulib too.
>
> The usefulness of this step depends on how much it would reduce the
> frequency of the x86_64 runs (which currently are at 1/week). Most
> parts of Gnulib are not arch-specific; therefore I think the minutes
> are better invested in testing Alpine Linux, FreeBSD, OpenBSD, than
> arm64/ppc64el.

Yes having more OSes is a good first step, but then having more
architectures than amd64 becomes relevant.

/Simon

signature.asc
Description: PGP signature