RE: What is a 'program' in Section 15.3.3.1?

[Arthur Schwarz]

Thanks.

(Also see lists.gnu.org/archive/html/automake/2015-05/msg00051.html How are
data files input for to custom drivers?)

> The TAP interface is very different.  

TAP is supposed to be a custom test driver. Is the interface and manner of
calling different from other custom drivers, and the API and other comments
describing custom drivers in the manual?

> 
> It depends on subordinate 
> scripts printing 

I'm very confused here. It looks to me like tap-driver.sh is a standalone
script and doesn't need any help scripts. The input data is processed in awk
and all the needed functions are defined in this context.


> 
> For GraphicsMagick, I created a script which is sourced by actual test 
> scripts (".tap" scripts), which provides generic functions used by 
> those scripts.  The functions support pass/fail detection based on 
> program exit status, but also support 'xfail' by being passed a set of 
> feature id strings which are matched which the actual available 
> features.  If the test fails, but a necessary feature is missing, then 
> an extra diagnosis is added which Automake's TAP script will turn into 
> an informative XFAIL message:

I am confused. Using Automake the Developer can generate a reference to a
class of test cases defined in the TESTS variable. Each one of the scripts
is required to output the results of one of more tests that they run in
quasi-TAP format. The TAP script, tap-driver.sh, takes the output and
generates a trs file. Included with the tap-driver.sh there is a means to
generate XFAIL and XPASS, however this seems to be global to all subtests
executed in a TESTS defined test. Each ok or not ok returned will be
translated (as required) to XPASS or XFAIL depending on
--expect-failure=yes.

As an example:
TEST_EXTENSIONS =.abc
ABC_LOG_DRIVER = tap-driver.sh
TESTS = test1.abc test2.abc

Means that tap-driver.sh is called twice, once for test1.abc, once for
test2.abc. Each of these tests can return one or more of ok, not ok, skip,
bail, or a YAML-like text string defined by --diagnostic-string and
--comments. 

Another point of confusion is that the options defined for the custom test
drivers have the format "a=b" if 'a' has a value, but tap-driver.sh does not
allow "a=b", it requires "a b".

Anyhow, thanks for the heads up, I will be looking at your files. And, as
always, I am confused.
>