Re: Continuous integration

[Phillip Lord]

Toon Claes <address@hidden> writes:

> address@hidden (Phillip Lord) writes:
>
>> I have the buildbot installation up now. Slightly harder work than I
>> hoped, but not too bad.
>>
>> http://emacs.bioswarm.net:8010
>
> This file probably can be improved in many ways, but I got a successful
> build. You can visit the build log here:
>
> https://gitlab.com/emacs-ci/emacs/builds/13595493

It's very nice, I think nicer that buildbot, especially as it seems to
have got branches sorted out properly.


>> Currently, it's running a single build (full build from clean, through
>> to tests). It will build any branch (following a change). The build
>> takes about 60 mins (or 30 mins with parallel builds). In practice, I'd
>> probably add a "incremental recompile and test" job which would be much
>> quicker. The builds are running on the master which is probably not
>> ideal.
>
> That's great work Phil! I still have to figure out everything it does,
> but it seems to be very comprehensive.
>
> The set up at GitLab.com is doing quite the same at the moment. Doing
> incremental recompilation would be quite hard on GitLab, because each
> build is done in a clean Docker container, so you'll have to export
> artifacts and reuse them each time.
>
> GitLab has a feature called pipelines, which allows you to chain builds
> together in stages. So this could be an example pipeline:
>
> test --> build some GNU/Linux distro
>      \
>       -> build macOS
>      \
>       -> build Windows
>
> The build stages won't be executed if the test stage failed.
> If I understand it correctly, buildbot does something similar?

I can do anything I like with buildbot. It's programmatic. The flip side
is, of course, you have to program it; it's not very declarative.


> At the moment I only have configured 1 stage on GitLab, because only 1
> was needed at the moment. The build stages shown in the flowchart above
> can be added in a later stage to make regular builds for different
> platforms automatically.
>
> At first sight, also the concept of Workers on buildbot (called Runners
> on GitLab) are quite similar.
>
> What I do not yet understand is what the Builders are, and what the
> difference is between full and quick?


"Builders" are different ways of building things. So, you might have one
specific to windows, or one which runs tests and one which does not.

I've tried to set it up to do an incremental build (that's what "quick"
is). But, I haven't done it right. It's building all branches, but
doesn't understand them. So it's doing subsequent incremental builds on
different branches in the same working directory which is causing the
sorts of breakages that you would expect. Having incremental builds
working is kind of important, I think because the bootstrap is so
slow. Working out how to clean them next time after a failure is
important though.

Phil