Re: Where is the testing? (was: Re: [PATCH] fs/xfs: Avoid unreadble filesystem if V4 superblock)

[Daniel Kiper]

On Wed, Sep 08, 2021 at 01:22:20AM +0000, Glenn Washburn wrote:
> On Thu, 2 Sep 2021 10:56:49 +0200
> Carlos Maiolino <cmaiolino@redhat.com> wrote:
>
> > On Wed, Sep 01, 2021 at 02:40:57PM +0200, Daniel Kiper wrote:
> > > CC-ing Javier...
> > >
> > > On Mon, Aug 30, 2021 at 01:48:50PM +0200, Carlos Maiolino wrote:
> > > > Hi.
> > > > On Mon, Aug 30, 2021 at 11:18:31AM +0200, Erwan Velu wrote:
> > > > >    Good day list,
> > > > >    Le jeu. 26 août 2021 à 15:26, Carlos Maiolino
> > > > > <[1]cmaiolino@redhat.com> a écrit :
> > > > >
> > > > >      [..]
> > > > >      Thanks for spotting this!
> > > > >
> > > > >    I'm adding the maintainers in CC. Carlos who commit the
> > > > > patch I'm fixing, agreed on the content.
> > > >
> > > > I didn't test the patch itself yet, but I've reproduced the
> > > > issue. I was quite sure I had tested this patch on a V4 fs, but
> > > > looks like I miscalculated the sizing. Thanks again. I'll try to
> > > > test the patch here asap.
> > >
> > > Did you test this patch? If yes may I add your Tested-by to it?
> >
> > Yup, patch works fine, just finished testing it, I was just trying to
> > understand where/why I miscalculated the inode size on V4
> > filesystems, and the reason was the same why Erwan split the
> > last/first members of inode v2/v3 in two different unused structs.
> >
> > Feel free to add to the patch:
> >
> > Tested-by: Carlos Maiolino <cmaiolino@redhat.com>
>
> It looks like the xfs_test test succeeds with tag grub-2.06-rc1a, fails
> with tag grub-2.06, and succeeds with current master. Yes, as expected.
> However, what this tells me is that no "make check" tests were done
> before finalizing the 2.06 release. I was under the impression that that
> was part of the release procedure. If its not, it would seem that we're
> not using the tests at a time when they would have the most impact.

Currently I run build tests for all architectures and platforms and
Coverity for x86_64 EFI before every push and release. I know this is
not enough but I tried "make check" at least once and got the impression
that the tests are not reliable. Sadly I have not time to dive deeper
and fix this and that. If someone, you?, wants to verify all tests and
fix broken ones I will be more than happy to start using it (it seems to
me your "[PATCH v2 0/8] Various fixes/improvements for tests" patch set
fixes some issues but I am not sure if all of them).

> It is my understanding that we have travis-ci tests that get run (at
> some point?), however they are only build tests and so would not have
> caught this. It was precisely this scenario that I hoped to avoid by
> doing more thorough continuous integration, which runs the extensive
> array of "make check" tests, when I submitted the Gitlab-CI patch
> series (which would've caught this automatically if it had been merged).
> To me this scenario is the poster child for taking action on this
> front. Can we make this a priority?

I think I can take a closer look at patch set mentioned above in a week
or two. If "make check" works as expected and can be run locally then we
can think about automating the testing.

Daniel