Re: [lmi] Headless GUI tests

[Greg Chicares]

On 2020-10-15 20:33, Vadim Zeitlin wrote:
> On Wed, 14 Oct 2020 22:51:40 +0000 Greg Chicares <gchicares@sbcglobal.net> 
> wrote:
> 
> GC> I try running the GUI test, both with and without xvfb:
[...]
> GC> /opt/lmi/src/lmi[0]$time xvfb-run ./gui_test.sh
[...about 100 seconds, vs. about 40 seconds without xvfb-run...]
> GC> Apparently it's working fine, except that when the test
> GC> completes, 'xvfb-run' seems to wait one minute and then
> GC> shut down, issuing a "shutdown" diagnostic twice.
> 
>  I think I know why do you get these error messages: I believe that the X
> server connection is actually opened not by wine itself, but by wineserver
> which is launches. But xvfb-run exits once wine command itself does, while
> wineserver exits slightly later, so it loses its connection to the server
> because it exists earlier.

Sounds like a reasonable explanation.

>  This is probably harmless, but the messages are, of course, annoying. To
> get rid of them we would need to either use a helper script, which would
> run "wine whatever" first and then wait until wineserver exits too before
> exiting itself and run this script under xvfb-run. Or avoid using xvfb-run
> and have our own wine-xvfb-run script which would do the same thing as
> xvfb-run does, i.e. start Xvfb server, launch the program, but then wait
> until both it and wineserver exit before killing Xvfb server. I don't know
> which one of these approaches is preferable/less ugly, but AFAICS they
> should both work.

Using Xvfb instead still takes 100 seconds instead of 40:

$Xvfb :1 -screen 0 1280x960x24 &                                      
[1] 3687
$time DISPLAY=:1 ./gui_test.sh                                       
NOTE: starting the test suite
SUCCESS: 21 tests successfully completed.
NOTE: 4 tests were skipped
DISPLAY=:1 ./gui_test.sh  27.68s user 9.13s system 36% cpu 1:39.78 total
$jobs -l                                                                    
[1]  + 3687 running    Xvfb :1 -screen 0 1280x960x24
$kill 3687
$
[1]  + done       Xvfb :1 -screen 0 1280x960x24

so the one-minute delay isn't attributable to the xvfb-run script.

>  But I don't really know where does the one minute delay come from. This is
> probably something specific to the GUI test, as I don't see it with a
> simple example (unlike the error messages that I do see perfectly well).

Yes, I see exactly the same here.

> I'll try to debug this later, unless you tell me not to (e.g. because
> you've decided to use Xpra or because you've already found the reason for
> it yourself).

Please do--you have more experience, and I'm at a dead end here.

> GC> $time xvfb-run ./gui_test.sh                                    
> GC> NOTE: starting the test suite
> GC> Abnormal-termination handler called. Please report this problem.
> 
>  Should I/we look into this or are you already looking or have already
> found the source of this problem?

TL;DR: Yes please.

I had hoped that I could fix this little nuisance easily, and then
invoke wx_test from nychthemeral_test, as I have wished for so long;
and then turn to some pressing work. But my ambition to clean up
nuisances exceeded my expertise in this and a couple other cases,
so I'd be very glad if you'd look into this.

> GC> Workaround #2: Because 'xvfb' doesn't seem to be working ideally, I
> GC> looked into 'xpra' as an alternative;
> 
>  This is very useful too, but I'd be more interested in making xvfb work,
> personally, as this would allow us to run GUI tests in GitHub CI builds
> too, while Xpra can't be used there.

That's a compelling reason to abandon Xpra and use xvfb instead.