Re: [Bug-gnubg] Big problems with external player

[Joern Thyssen]

On Thu, May 06, 2004 at 06:27:48PM +0200, Joachim Matussek wrote
> Hello,
> 
> i am comparing several match equity tables by 2 GNUBG nogui instances playing 
> each other.
> 
> First problem is... i tried several new versions of GNUBG nogui (1578 now). 
> None of them works as external player on my new tower pc. Although they work 
> fine with my notebook. Both computers use Windows 2000, SP 4, all updates. At 
> last i found a version that works with my tower pc. It is a build created by 
> Holger in november 2003. 
> 
> Second problem is... the match series is done by a python script
> provided by Martin Janke. It is the same script Achim Mueller used.
> Martin Janke told me the output at the python interface is permuted.
> This means if the output says "player 0 wins" in fact "player 1 wins"
> and vice versa. He didn´t correct this in his script what seems quite
> reasonable. 

In gnubg: player 0 (zero) = O (oh) = top player
          player 1        = X      = bottom player

In python: 'X' = player 0
           'O' = player 1

Achim's python script has:

if game['info']['winner'] == 'O':
   ...
   winner = 0

so it appears that you should invert the result.

> 
> Third problem is... i am doing sessions of 100.000 trials 0-ply, match
> length 15. I used several match equity tables (Woolsey - Snowie,
> Snowie - Woolsey, Mec26 - Snowie, Snowie - Mec26 ...). I did at least
> 8 sessions. All the time player 1 (means player 0) was the winner by
> about 50,4 % against 49,6 %, no matter what match equity table was
> used or what permutation. I feel there must be some bias either
> introduced by my experiment or by some GNUBG bug.

Using the scripts I just did 1,000 3pt matches of woolsey versus snowie:

External player = config0 = Snowie MET = Player 1 = '0' in statsPath = 51.0%
"Internal" player = config1 = Woolsey MET = Player 0 = '1' in statsPath = 49.0%

The confidence interval is 4.5%.

I checked a few of the .sgf files against the written statsPath, and
everything seems consistent.

I've modified the scripts slightly:

address@hidden gnubg-head4 87]$ cat config0
set matchequitytable "met/snowie.xml"
set evaluation chequerplay eval plies 0
set evaluation cubed eval plies 0
external localhost:10000

(the external player uses the eval settings specified by "set eval")

address@hidden gnubg-head4 91]$ cat config1
set matchequitytable "met/woolsey.xml"
set player 1 gnu
set player 1 chequerplay evaluation plies 0
set player 1 cubedecision evaluation plies 0
set player 0 external localhost:10000
>
from matchSeries import *
playMatchSeries (matchLength = 3, noOfMatches = 1000,
                 statsFile = "statsPath", sgfBasePath = None,
                 matBasePath = None)

(the '>' makes gnubg go into python)

Joern