Re: [lmi] Headless GUI tests

[Greg Chicares]

On 11/14/21 8:11 PM, Vadim Zeitlin wrote:
[...]
>  IOW I think I have conclusively demonstrated that what you're seeing is
> impossible. Knowing that it clearly is, where is the mistake in my
> reasoning?

I can't identify the cause of this problem by global reasoning,
so I'm trying to localize it empirically first, hoping to produce
a smaller problem that's easier to reason about.

For example, in the first experimental patch below [0], the first
hunk (if not commented out) causes the test to succeed. A new
census window is created, then closed, and that just works.
Of course, that prevents the test from actually testing anything,
but at least we've established an empirical fact.

In the second hunk (the first one not commented out), the census
has been modified, and already I can't find a technique to close
it without any drama. I figured that closing it would cause a
"Do you want to save changes" dialog to pop up, which I hoped to
close by sending an alt-n keystroke. That didn't work, so I also
tried "wxExpectModal<wxMessageDialog>(wxNO);", but that didn't
work either. I keep seeing:

  Assertion '"Assert failure"' failed
  (A message box "Do you want to save changes to unnamed1?" was shown 
unexpectedly.).
  [file /opt/lmi/local/include/wx-3.1/wx/testing.h, line 426]

Stepping back, I undo that first patch, and instead apply
the second experimental patch below [1]; now the program
does terminate on its own (good!), although the test fails:

xvfb-run --server-args="-fbdir /var/tmp" wine /opt/lmi/bin/wx_test.exe 
--ash_nazg  --data_path=/opt/lmi/data --test paste_census
NOTE: starting the test suite
paste_census: started
paste_census: ERROR (Assertion failure: Expected class defaults dialog was not 
shown. [file /opt/lmi/src/lmi/wx_test_paste_census.cpp, line 366, in run()].)
time=223ms (for all tests)
FAILURE: 1 out of 1 test failed.
X connection to :99 broken (explicit kill or server shutdown).                  
             

But at least now we've narrowed it down to a possible
cause. I.e., we try to pop up a dialog with Ctrl-Alt-e,
and when we test whether that worked, we find that it
apparently didn't. This is the only Ctrl-Alt-whatever
action in the whole GUI test. Is it possible that this
admittedly uncommon key-combination is eaten by someone
in the Xvfb-wine chain? If so, could that be a defect
in that combination of tools, which has been fixed in
a later version, explaining why you don't see this
failure but I do?

I don't see any other way to bring up this dialog,
because it has no other accelerator. I.e., if I pull
down the menu, no letter in the left-hand text for
this item is highlighted. Perhaps that's an lmi
defect that should be fixed anyway.

---------

[0] "the experimental patch below":

diff --git a/wx_test_paste_census.cpp b/wx_test_paste_census.cpp
index 27b73863e..d86486e04 100644
--- a/wx_test_paste_census.cpp
+++ b/wx_test_paste_census.cpp
@@ -235,11 +235,22 @@ LMI_WX_TEST_CASE(paste_census)
     // Create a new census.
     wx_test_new_census census;
 
+// Succeeds upon early exit here:
+// census.close();
+// return;
+
     // Paste data into it.
     wxUIActionSimulator ui;
     ui.Char('s', wxMOD_CONTROL | wxMOD_SHIFT); // "Census | Paste census"
     wxYield();
 
+census.close();
+wxYield();
+ui.Char('n', wxMOD_ALT); // "No" to "save changes to unnamed?"
+wxYield();
+wxExpectModal<wxMessageDialog>(wxNO);
+return;
+
     // Find the model containing the cells and check that it was filled in

[1] "the second experimental patch below"

diff --git a/wx_test_paste_census.cpp b/wx_test_paste_census.cpp
index 27b73863e..a7706b082 100644
--- a/wx_test_paste_census.cpp
+++ b/wx_test_paste_census.cpp
@@ -41,6 +41,7 @@
 
 #include <algorithm>
 #include <cstring>
+#include <iostream>
 #include <set>
 #include <sstream>
 #include <stdexcept>
@@ -235,11 +236,43 @@ LMI_WX_TEST_CASE(paste_census)
     // Create a new census.
     wx_test_new_census census;
 
+// Succeeds upon early exit here:
+// census.close();
+// return;
+
     // Paste data into it.
     wxUIActionSimulator ui;
     ui.Char('s', wxMOD_CONTROL | wxMOD_SHIFT); // "Census | Paste census"
     wxYield();
 
+// These ideas don't work--a wx assertion failure still occurs.
+// std::cout << __LINE__ << std::endl;
+// census.close();
+// std::cout << __LINE__ << std::endl;
+// wxYield();
+// ui.Char('n', wxMOD_ALT); // "No" to "save changes to unnamed?"
+// std::cout << __LINE__ << std::endl;
+// wxYield();
+// wxExpectModal<wxMessageDialog>(wxNO);
+// std::cout << __LINE__ << std::endl;
+// wxYield();
+// std::cout << __LINE__ << std::endl;
+// return;
+
+// Saving the census file now, as soon as it has been modified,
+// is a temporary expedient that prevents a
+//   A message box "Do you want to save changes to unnamed1?" was shown 
unexpectedly.
+// messagebox, which causes the program not to exit without a
+// manual "^C".
+    ui.Char('a', wxMOD_CONTROL);    // "File|Save as"
+    wxTEST_DIALOG
+        (wxYield()
+        ,wxExpectModal<wxFileDialog>("eraseme").
+            Describe("census save file dialog")
+        );
+// census.close();
+// return;
+
     // Find the model containing the cells and check that it was filled in
     // correctly.
     wxGrid* const grid_window = find_census_grid_window();
@@ -264,11 +297,28 @@ LMI_WX_TEST_CASE(paste_census)
             )
         );
     ui.MouseClick();
+    grid_window->SelectRow(0);
     wxYield();
 
     LMI_ASSERT_EQUAL(lmi::ssize(grid_window->GetSelectedRows()), 1);
 
+// census.close();
+// return;
+
     ui.Char('e', wxMOD_CONTROL | wxMOD_ALT); // "Census|Edit class defaults"
+    // That 'ctrl-alt-e' should have brought up a class-defaults
+    // dialog, but apparently it didn't, as an assertion fails
+    // below, complaining:
+    //   "Expected class defaults dialog was not shown."
+    // Explicitly yielding here:
+    wxYield();
+    // does not prevent that failure.
+    //
+    // This is the only instance in the GUI test of either of these regexes:
+    //   git grep 'CONTROL.*ALT'
+    //   git grep 'ALT.*CONTROL'
+    // so is it possible that ctrl-alt-alpha keystrokes just
+    // don't work with Xvfb+wine?
 
     struct change_gender_in_class_defaults_dialog
         :public wxExpectModalBase<MvcController>