Re: [RFC] Enabling GitLab CI

[James Lowe]

On 18/05/2020 10:29, Jonas Hahnfeld wrote:
> Am Sonntag, den 17.05.2020, 21:14 +0100 schrieb James Lowe:
> > On 17/05/2020 20:27, Jonas Hahnfeld wrote:
> > >    - Comparison of regression tests is not yet integrated, the main
> > > problem being the need for a baseline. I already have an idea or two
> > > how this could work, but for now I'm focusing on the initial setup.
> > > This means James still needs to download the patches and run 'make
> > > check' (make doc being run automatically).
> > Yes that would save me time.
> >
> > Something that strikes me immediately.
> >
> > * If a CI run makes doc are we going to have an entry and or label in
> > the comment for a success/failure?
> >
> > I don't want to bother with any test if make/make doc fails, so I assume
> > that it will change the label to take it off patch new. But at the same
> > time if a patch is 'new' and hasn't had the make/make doc test, do I
> > still run the make check anyway? If so then it passed the label would go
> > to review and I assume the CI run would still catch that and do the
> > make/make doc. Otherwise I am going to not only have to check for a
> > patch 'new' I am going to have to know that it is waiting on a make doc
> > first if you see what I mean? (will I have to second guess the CI so to
> > speak)
> I'm not sure I fully understand the question(s).
>
> First of all, the system tests every new commit added to a merge
> request, no matter how small or if the diff is the same as before.

Yes but make doc isn't instantaneous it takes time .. a good amount of 
time. No matter how small the diff is.

> (That's the advantage of computers, they don't get bored at doing the
> same over and over again.) If the tests pass, you get nice green ticks
> everywhere. On error GitLab shows red crosses instead, in particular
> the stage that failed (build, test, doc). This currently doesn't add
> the Patch::needs_work label, but I can certainly make our bot do this.

Yes please, I need a simple way to know not even to bother testing a 
make check because if make doc fails the entire patch is useless. In 
some cases stuff that breaks make doc also would break 'make' (when 
generating our 'info' pages, some doc edits if not done correctly 
manifest themselves for just make rather than breaking only 'make doc).

> As the pipelines start immediately after the push, they might already
> be finished when you look at the merge request. If something didn't
> pass, I think it's perfectly ok to skip manual tests.
but how do I know? That is the nub of what I am asking. If a patch is 
'new' how do I know that an automated make doc is 'in progress, has 
completed with errors, has completed without errors' as I am not going 
to bother to do any work on it if it is the first two cases. What is the 
point?
> In case the pipelines are still running or even pending, I think you
> should ignore that merge request for the time being and maybe come back
> later. We'll see how much of an issue this turns out to be.

It will be an issue. Unless it is easy for me to check.


Imagine you sit down and see 20 patch-new merge requests to test.

Which one do you do first?

Which one has already had its make doc run?

Which one has failed/passed/is in progress of its make doc run?


I hope that is clearer.

James