Re: [certi-dev] How to contribute to HLA TestSuite

[Eric Noulard]

2009/8/13 Gotthard, Petr <address@hidden>:
> Hi Eric,
>
> One comment: Would it be possible to prepare a simple dtest template-- two 
> "empty" federates that
>I could copy and update in order to create new a test for this suite?

Yes good idea.

> The existing HLA TestSuite tests are quite complex to understand how dtest 
> works and how to write custom multi-federate tests.

In fact if you look at current dtest scripts in HLA test suite they are
already pretty simple.
However I do agree they may be hard to read without "teacher aside".
The dtest script deals with the same thing every time:

1) handle command line parameters and defaults values
2) create RTIG DTester and it's associated sequence (which is almost
always the same)
3) create as many DTester object as you have federates
4) write the sequence for each federate
5) register the DTesters and run the test.

Here comes the current cheat sheet (example taken from dtest_test_Sync.py):

1) is not varying much (unless you have specific parameter to pass to your test)

def usage():
    print "Usage:\n %s [--help] [--certi_home=<path>]
--rtig=[[<user>@]<host>]:<rtig_path>
--federate=[[<user>@]<host>]:<federate_path>" % sys.argv[0]

try:
    opts, args = getopt.getopt(sys.argv[1:], "hr:f:c:",
["help","rtig=", "federate=","certi_home="])
except getopt.GetoptError, err:
    print >> sys.stderr, "opt = %s, msg = %s" % (err.opt,err.msg)
    usage()
    sys.exit(2)

## default values
certi_home_defined=False
rtig_param = dtest.Utils.getUserHostPath("rtig")
federate_param = dtest.Utils.getUserHostPath("test_Sync")
federate_param['fom']="test_Sync.fed"

for o, a in opts:
    if o in ("--help"):
            usage()
            sys.exit(2)
    if o in ("-r", "--rtig"):
        rtig_param   = dtest.Utils.getUserHostPath(a)
    if o in ("-f", "--federate"):
        federate_param = dtest.Utils.getUserHostPath(a)
    if o in ("-c", "--certi_home"):
        certi_home = a
        certi_home_defined=True

if not certi_home_defined:
    if os.environ.has_key("CERTI_HOME"):
        certi_home=os.environ["CERTI_HOME"]
    else:
        print "You must define CERTI_HOME environment variable"
        sys.exit(2)

2) is almost always the same

  rtig = dtest.DTester("RTIG",

session=dtest.SSHSessionHandler(rtig_param['user'],host=rtig_param['host']))

# you may change the default time out value
rtig.timeout = 40
# you add want to save the output of your dtester to a file.
rtig.stdout    = file(rtig.name + ".out",'w+')
rtig.stdin     = file(rtig.name + ".in",'w+')
rtig.stderr    = file(rtig.name + ".err",'w+')

# describe RTIG run steps
rtig.addRunStep("ok",True,"HLA test test_Sync Starts.")
dtest.ReusableSequences.addConditionalRunShellScript(rtig,c_shell_cmd="source
"+certi_home+"/share/scripts/myCERTI_env.csh "+rtig_param['host'],
                               bourne_shell_cmd="source
"+certi_home+"/share/scripts/myCERTI_env.sh "+rtig_param['host'])
rtig.addRunStep("runCommand",command=rtig_param['path'])
rtig.addRunStep("expectFromCommand",pattern="CERTI RTIG up and
running",timeout=5)
rtig.addRunStep("barrier","RTIG started")
rtig.addRunStep("barrier","All Federate(s) ended")
rtig.addRunStep("terminateCommand")
rtig.addRunStep("waitCommandTermination")
rtig.addRunStep("ok",True,"HLA test test_Sync Ends.")

3)  Is one line per Federate

firstFederate = dtest.DTester("test_Sync_First",

session=dtest.SSHSessionHandler(federate_param['user'],host=federate_param['host']))

otherFederate = dtest.DTester("test_Sync_Other1",

session=dtest.SSHSessionHandler(federate_param['user'],host=federate_param['host']))

4) This steps is the heart of your test.

5) Is not varying much.

def goTest():
    myDTestMaster = dtest.DTestMaster("HLA test test_Sync
Starts","Launch RTIG + two test_Sync federate for testing
synchronization point and callback,...")
    myDTestMaster.timeout = 40
    myDTestMaster.register(rtig)
    myDTestMaster.register(firstFederate)
    myDTestMaster.register(otherFederate)
    myDTestMaster.startTestSequence()
    myDTestMaster.waitTestSequenceEnd()

goTest()

> There may be more people with a similar problem,
> so this could help to overcome the "initial resistance" of dtest. ;-)

What I can propose is the following, you write the dummy C++ code for
the dummy federate  and I write the over-commented dtest-script for that.

The dummy federate may simply create/join/resign/destroy the federation.
The only needed feature of such a federate with current HLA TestsSuite tools
is:

   1) The federate must print some output to stdout when succeeding it's task
       such that the output may be scanned by dtest in order to
validate the test.

  2) There should be a single federate executable which can play different role
      if given a specified flag. i.e. if you want to build a test with
3 federates
      using dummy_federate executable which handle this
         dummy_federate --role="viewer"
         dummy_federate --role="client"
         dummy_federate --role="server"

      instead of having 3 separate executables:
            dummy_federate_viewer
            dummy_federate_client
            dummy_federate_server

        This is not a DTest limitation, but the current
HLATestsSuiteTools.cmake
        which provides the HLATestsSuite_ADD_TEST macro is not smart
        enough to handle several executables.
        (we could change that but there is some work).


In order to reduce the size of the dtest scripts I may try to factor out the
common part (like the RTIG DTester definition) in a separate  python file
that may be used by actual dtest scripts.

What do you think?
How can I help?


-- 
Erk
Membre de l'April - « promouvoir et défendre le logiciel libre » -
http://www.april.org