Re: [lmi] Headless GUI tests

[Vadim Zeitlin]

On Mon, 15 Nov 2021 01:13:48 +0000 Greg Chicares <gchicares@sbcglobal.net> 
wrote:

GC> With your patch
GC>   https://github.com/let-me-illustrate/lmi/pull/197

 (Thanks for applying this)

GC> now I observe something a bit different:

 I still don't see anything like this, i.e. for me it's still just

NOTE: starting the test suite
paste_census: started
time=7565ms (for paste_census)
paste_census: ok
time=7566ms (for all tests)
SUCCESS: 1 test successfully completed.

GC> >  Unfortunately I still can't reproduce the problem,
GC> 
GC> Perhaps it requires this exact version of 'wine':
GC>   $wine --version
GC>   wine-4.0.3 (Debian 4.0.3-1)
GC> I can't think of any other dependency that seems likely to account for
GC> the difference in what we're seeing.

 I agree that the difference in Wine version could be important, and I did
test with Wine 5.0.3, which is the version currently available in all of
Bullseye, Bookworm and Sid. 4.0.3 is a very strange version, as Buster is
supposed to have 4.0.2 (see https://packages.debian.org/buster/wine), so I
don't even know where could I get it from. I could build it myself, of
course, but I've only ever built it from the pristine upstream sources and
not from the Debian package.

 Before looking further into it, I'd like to confirm that it's really
normal and expected that you're using Buster (in first approximation) Wine
version on your Bullseye system?

GC> I haven't found a way to prevent it, but I can get a different
GC> outcome with this patch (which should at least do no harm: if
GC> the mouse click selects row zero, then SelectRow(0) should have
GC> no observable effect):
GC> 
GC> --8<---8<---8<---8<---8<---8<---8<---8<---8<---8<---8<---
GC> diff --git a/wx_test_paste_census.cpp b/wx_test_paste_census.cpp
GC> index 27b73863e..a09afc3c1 100644
GC> --- a/wx_test_paste_census.cpp
GC> +++ b/wx_test_paste_census.cpp
GC> @@ -264,6 +264,7 @@ LMI_WX_TEST_CASE(paste_census)
GC>              )
GC>          );
GC>      ui.MouseClick();
GC> +    grid_window->SelectRow(0);
GC>      wxYield();
GC>  
GC>      LMI_ASSERT_EQUAL(lmi::ssize(grid_window->GetSelectedRows()), 1);
GC> --8<---8<---8<---8<---8<---8<---8<---8<---8<---8<---8<---

 Needless to say, this doesn't change anything for me and the test still
passes :-(

On Mon, 15 Nov 2021 15:47:16 +0000 Greg Chicares <gchicares@sbcglobal.net> 
wrote:

GC> On 11/14/21 8:11 PM, Vadim Zeitlin wrote:
GC> [...]
GC> >  IOW I think I have conclusively demonstrated that what you're seeing is
GC> > impossible. Knowing that it clearly is, where is the mistake in my
GC> > reasoning?
GC> 
GC> I can't identify the cause of this problem by global reasoning,
GC> so I'm trying to localize it empirically first, hoping to produce
GC> a smaller problem that's easier to reason about.
GC> 
GC> For example, in the first experimental patch below [0], the first
GC> hunk (if not commented out) causes the test to succeed. A new
GC> census window is created, then closed, and that just works.
GC> Of course, that prevents the test from actually testing anything,
GC> but at least we've established an empirical fact.
GC> 
GC> In the second hunk (the first one not commented out), the census
GC> has been modified, and already I can't find a technique to close
GC> it without any drama. I figured that closing it would cause a
GC> "Do you want to save changes" dialog to pop up, which I hoped to
GC> close by sending an alt-n keystroke. That didn't work,

 No, this can't really work because the dialog is not opened yet when you
send it. This is why we need wxExpectModal to deal with the modal dialogs
before they open.

GC> so I also tried "wxExpectModal<wxMessageDialog>(wxNO);", but that
GC> didn't work either.

 You can't just use it like you did, i.e.

        +wxYield();
        +wxExpectModal<wxMessageDialog>(wxNO);
        +return;

as this has no effect. You need to use it inside wxTEST_DIALOG() macro, as
wx_test_document_base dtor does.

GC> Stepping back, I undo that first patch, and instead apply
GC> the second experimental patch below [1]; now the program
GC> does terminate on its own (good!), although the test fails:
GC> 
GC> xvfb-run --server-args="-fbdir /var/tmp" wine /opt/lmi/bin/wx_test.exe 
--ash_nazg  --data_path=/opt/lmi/data --test paste_census
GC> NOTE: starting the test suite
GC> paste_census: started
GC> paste_census: ERROR (Assertion failure: Expected class defaults dialog was 
not shown. [file /opt/lmi/src/lmi/wx_test_paste_census.cpp, line 366, in 
run()].)
GC> time=223ms (for all tests)
GC> FAILURE: 1 out of 1 test failed.
GC> X connection to :99 broken (explicit kill or server shutdown).              
                 
GC> 
GC> But at least now we've narrowed it down to a possible
GC> cause. I.e., we try to pop up a dialog with Ctrl-Alt-e,
GC> and when we test whether that worked, we find that it
GC> apparently didn't. This is the only Ctrl-Alt-whatever
GC> action in the whole GUI test. Is it possible that this
GC> admittedly uncommon key-combination is eaten by someone
GC> in the Xvfb-wine chain?

 It seems more plausible that this key combination is somehow not mapped
correctly, e.g. Alt is not the expected modifier (in this version of Wine)
or something like this. But this would be easy to check: couldn't you
temporarily use some other accelerator, not involving Alt, for this menu
command and in the test? If this suffices to solve the problem, it would
confirm that this is indeed the culprit.

GC> I don't see any other way to bring up this dialog,
GC> because it has no other accelerator. I.e., if I pull
GC> down the menu, no letter in the left-hand text for
GC> this item is highlighted.

 It should also be possible to simulate (down) arrow key and then "Enter"
to select it, but using accelerators is, of course, much more convenient.

 Anyhow, from my point of view, the real problem itself might actually not
be so important, if it only happens due to Wine 4.0 mishandling Ctrl-Alt-E
somehow. But I still don't understand how do we end up with a dialog,
instead of "just" an assert failure, which is supposed to happen due to the
existence of "expect_no_dialogs" in the outermost scope.

 Please let me know if you have any ideas about this, thanks in advance,
VZ

pgpIx3BYjCOhs.pgp
Description: PGP signature