Re: Emacs 27.1 Windows Binaries -- testing wanted

[phillip . lord]

On 2020-08-22 14:08, Eli Zaretskii wrote:
> > Date: Sat, 22 Aug 2020 13:52:22 +0100
> > From: phillip.lord@russet.org.uk
> > Cc: emacs-devel@gnu.org
> >
> > > I'm not talking about using ert or not, I'm talking about determining
> > > which of the features should check out and which shouldn't.  Are you
> > > going to edit w32-features.el by hand each time you add or remove an
> > > optional library from the build?  Or will the list of the features
> > > which should check out be constructed automagically somehow?
> >
> > It would be manual. The list of features changes rarely enough that 
> > it's
> > hard to automate this and know that it will work. The automation is as
> > likely to break as a manual system
>
> I don't understand why it must break, if you base it on
> system-configuration-features, or on some new file, to be generated by
> the build procedure.  Did you consider one of these alternatives?  If
> yes, why did you reject them?

I didn't consider it much. I cannot see how to go from this to something 
I can test. Say, `system-configuration-features` lists "XPM", I cannot 
turn that into something I could test.

I could modify by test something like

(ert-deftest
  (skip-unless (string-match-p "XPM" system-configuration-features))
  (should (image-type-available 'xpm))
)


> > If I think back to the bugs with Emacs binary releases, this would 
> > have
> > stopped only a few: harfbuzz and svg failure in this release. It would
> > not have prevented missing dependencies (I have missed all of 
> > harfbuzz,
> > lcms and libjansson till late in the day), nor the failed optimization
> > or stripping.
>
> I don't understand this, either.  If the dependencies of HarfBuzz are
> not installed, then the HarfBuzz font backend will not be available,
> and the way I suggested for testing its availability will tell you
> HarfBuzz doesn't work, which is AFAIU what you wanted.  And the same
> with any other DLLs: if their dependencies are not available, they
> will fail to load when the feature test runs, and the test will fail.

It would work if, when harfbuzz was installed w32-features was updated, 
but not if it wasn't. As the harfbuzz merge didn't update build-deps.zh 
which it arguable should have, then why would a test file be updated.


> > > If the latter, I didn't see you describe how will that happen.  If the
> > > former, it means a file in the distribution will depend on the details
> > > of how you build the official binaries, which could be different from
> > > what end-users do on their systems (those who build their own Emacs),
> > > so the tests will fail for them, AFAIU.
> >
> > That is true with all of our tests, I think.
>
> No, our tests use skip-unless to avoid running tests that need
> features or programs which are unavailable.
>
> > json parsing is tested by make check and it will not work on msys2
> > if the dependencies are not installed.
>
> I didn't look into this, but if json testing fails instead of skipping
> all the tests, that's a bug that should be fixed.

I think you are correct about all of these. In either case, the tests 
will not pass!


> > In this case, I am not suggesting adding ert tests to make check,
> > but a manually run test specifically for understanding the features
> > of a binary build. I am guessing anyone running this would be
> > understand what they are doing enough to know why. And I will
> > document the entry function.
>
> That's not what worries me.  What worries me is the fact that a file
> distributed as part of the Lisp libraries will have its contents
> depend on the last release you personally did, with whatever quirks
> that required from the tests in that file.  It doesn't sound right to
> me.


Not sure I understand this. A lisp library with "w32-" in it's name will 
have behaviour which reflects how the official binaries are supposed to 
behave. If this worries you, as I say, I can add it to `data-directory' 
instead. But, it has to be in the distribution.


> > > It might also mean problems in merging from the release branch to
> > > master,
> > > e.g. if we remove some optional library from master that is still being
> > > used on the release
> > > branch.
> >
> > That problem is there anyway in build-deps.py
>
> But build-deps.py is under admin, whereas you are suggesting to do
> this under lisp/.

Having a test functionality under lisp/ does, I agree, seem morally 
wrong. Likewise `data-directory'. But, the reasons for it are clear, and 
what harm would it cause in practice?


Phil