[lmi] Headless GUI tests [Was: GUI on corporate server?]

[Greg Chicares]

On 2020-10-13 22:03, Vadim Zeitlin wrote:
> On Tue, 13 Oct 2020 20:52:54 +0000 Greg Chicares <gchicares@sbcglobal.net> 
> wrote:
[...]
> GC>  - can't run lmi's own GUI under 'wine'
> 
>  I think both of these could be addressed by using xvfb. This is what I use
> for running GUI tests in the CI builds and it works reasonably well,
> although there are sometimes some weird problems that only happen under
> xvfb or, worse, that don't happen under it. But this is pretty rare. And
> using xvfb may be interesting even if you're using a real X server because
> you can run the tests inside it without interfering with your normal
> session, i.e. continue using your mouse and keyboard while they run.

After installing it:
  # apt-get install xvfb
I try running the GUI test, both with and without xvfb:

/opt/lmi/src/lmi[130]$time ./gui_test.sh         
NOTE: starting the test suite
SUCCESS: 21 tests successfully completed.
NOTE: 4 tests were skipped
./gui_test.sh  26.98s user 6.79s system 78% cpu 43.092 total

/opt/lmi/src/lmi[0]$time xvfb-run ./gui_test.sh
NOTE: starting the test suite
SUCCESS: 21 tests successfully completed.
NOTE: 4 tests were skipped
X connection to :99 broken (explicit kill or server shutdown).
X connection to :99 broken (explicit kill or server shutdown).
xvfb-run ./gui_test.sh  28.58s user 9.35s system 37% cpu 1:41.22 total

Apparently it's working fine, except that when the test
completes, 'xvfb-run' seems to wait one minute and then
shut down, issuing a "shutdown" diagnostic twice.

I tried inserting 'xvfb-run' into the shell script,
right before the 'wine' command:

--8<----8<----8<----8<----8<----8<----8<----8<----8<----8<--
diff --git a/gui_test.sh b/gui_test.sh
index 5175110e..3b0d43fb 100755
--- a/gui_test.sh
+++ b/gui_test.sh
@@ -153,5 +153,5 @@ mkdir --parents "$log_dir"
 
 cd "$srcdir"
 
-$PERFORM "$prefix"/bin/wx_test "$@" --ash_nazg --data_path="$prefix"/data 2>&1 
\
+xvfb-run $PERFORM "$prefix"/bin/wx_test "$@" --ash_nazg 
--data_path="$prefix"/data 2>&1 \
   | tee "$log_dir"/gui_test | sed -e "$gui_test_clutter"
--8<----8<----8<----8<----8<----8<----8<----8<----8<----8<--

and that prevents the "shutdown" diagnostic from appearing,
but the test still takes one minute longer when run headless:

/opt/lmi/src/lmi[0]$time ./gui_test.sh         
NOTE: starting the test suite
SUCCESS: 21 tests successfully completed.
NOTE: 4 tests were skipped
./gui_test.sh  28.52s user 8.94s system 37% cpu 1:40.57 total

and that delay still occurs if I run the command as
simply and directly as possible:

$time xvfb-run wine /opt/lmi/bin/wx_test --ash_nazg --data_path=/opt/lmi/data 
>/dev/null 2>&1
xvfb-run wine /opt/lmi/bin/wx_test --ash_nazg --data_path=/opt/lmi/data >  2>  
27.80s user 9.36s system 37% cpu 1:40.28 total

Then I tried two different ways to work around that delay,
but they both led in unexpected directions...

Workaround #1: I thought maybe it would work better with a
native build that doesn't need 'wine', but...

$make uninstall
$LMI_COMPILER=gcc ; LMI_TRIPLET=x86_64-pc-linux-gnu ; . 
/opt/lmi/src/lmi/set_toolchain.sh
$make install 2>&1 |less -S
$export LD_LIBRARY_PATH \
    LD_LIBRARY_PATH=. \
    LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/opt/lmi/local/"gcc_${LMI_TRIPLET}"/lib \
    LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/opt/lmi/bin
$git checkout -- gui_test.sh 
/opt/lmi/src/lmi[0]$time xvfb-run ./gui_test.sh
Alarum: Print directory '/opt/lmi/src/lmi/Z:/opt/lmi/print' not found.
[path_utility.cpp : 395]

Print directory '/opt/lmi/src/lmi/Z:/opt/lmi/print' not found.
[path_utility.cpp : 395]

Warning: If possible, data directory '/opt/lmi/data' will be used for print 
files instead.
[configurable_settings.cpp : 178]

^C
xvfb-run ./gui_test.sh  0.06s user 0.05s system 0% cpu 4:13.14 total

I interrupted that because it looked like it might run forever.
Just in case the invalid print directory was causing some problem,
I fixed 'configurable_settings.xml' manually and tried again:

$time xvfb-run ./gui_test.sh                                    
NOTE: starting the test suite
Abnormal-termination handler called. Please report this problem.
xvfb-run ./gui_test.sh  0.38s user 0.08s system 72% cpu 0.626 total

That's independent of xvfb:

$ /opt/lmi/bin/wx_test --ash_nazg --data_path=/opt/lmi/data           
NOTE: starting the test suite
about_dialog_version: started
Abnormal-termination handler called. Please report this problem.

but since it doesn't involve 'wine', I should be able to track
down the cause with gdb.

Workaround #2: Because 'xvfb' doesn't seem to be working ideally, I
looked into 'xpra' as an alternative; but when I tried to install it:

  (lmi_bullseye_3)root@ugolyok:/opt/lmi/src/lmi# apt-get install xpra
  Reading package lists... Done
  Building dependency tree       
  Reading state information... Done
  Package xpra is not available, but is referred to by another package.
  This may mean that the package is missing, has been obsoleted, or
  is only available from another source

  E: Package 'xpra' has no installation candidate

What puzzles me is that it's available for everything except bullseye:
  https://packages.debian.org/search?keywords=xpra
How can a package be in [old][old]stable, sid, and experimental,
but not in testing? I thought that if testing included a sid
version, and that version wasn't good enough, then the
maintainer would just revert testing to the stable version.