Re: patch to add AT_CHECK_EUNIT autotest macro

[Romain Lenglet]

Paolo Bonzini wrote:
>> The macro requires variables ERL, ERLC, and ERLCFLAGS to be defined. I
>> documented that those should be configured by configure, and I modified
>> the generated atconfig file to contain those configured values. It's
>> simple and transparent, but may be ugly since those will be defined for
>> all testsuites, even those that don't use AT_CHECK_EUNIT. Any suggestion
>> to make it better? And should I exit with code 99, in AT_CHECK_EUNIT, if
>> those variables are not defined?
> 
> No, probably with code 77 (skip the testcase if EUnit is not installed).

OK, I'll send an updated patch this weekend to do that.

>> Also, sorry for the form of this email.
>> I am sending this pach manually from a system without git.
> 
> This is not a problem.  Just ensure that the patch is attached as
> text/plain and not application/octet-stream, please.  (This is why I
> have not yet looked at it, even though I will later).

I switched back to a decent email client, so the attachment should be OK
this time.

Thanks,
--
Romain Lenglet

>From b6c3ba63960c28e8addfaafe3b0670cb2f9743b4 Mon Sep 17 00:00:00 2001
From: Romain Lenglet <address@hidden>
Date: Wed, 29 Jul 2009 21:25:21 +0900
Subject: [PATCH] Add AT_CHECK_EUNIT macro to run Erlang EUnit unittests, with 
its autotests and documentation.

---
 ChangeLog                |   13 +++++++
 NEWS                     |    5 +++
 doc/autoconf.texi        |   50 +++++++++++++++++++++++++
 lib/autoconf/autotest.m4 |   10 +++++-
 lib/autotest/Makefile.am |    2 +-
 lib/autotest/autotest.m4 |    1 +
 lib/autotest/erlang.m4   |   91 ++++++++++++++++++++++++++++++++++++++++++++++
 tests/autotest.at        |   74 +++++++++++++++++++++++++++++++++++++
 8 files changed, 244 insertions(+), 2 deletions(-)
 create mode 100644 lib/autotest/erlang.m4

diff --git a/ChangeLog b/ChangeLog
index 61bda1a..f38def7 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,16 @@
+2009-07-30  Romain Lenglet  <address@hidden>
+
+       * lib/autotest/erlang.m4 (AT_CHECK_EUNIT): New file defining new
+       AT_CHECK_EUNIT macro.
+       * lib/autotest/Makefile.am (dist_autotestlib_DATA): Add erlang.m4.
+       * lib/autoconf/autotest.m4 (AC_CONFIG_TESTDIR): Add definitions of
+       variables used by AT_CHECK_EUNIT macro: ERL, ERLC, ERLCFLAGS.
+       * tests/autotest.at (Erlang Eunit unittests): Add test for macro
+       AT_CHECK_EUNIT.
+       * doc/autoconf.texi (Writing Testsuites): Document macro
+       AT_CHECK_EUNIT.
+       * NEWS: Mention macro AT_CHECK_EUNIT.
+
 2009-07-29  Ralf Wildenhues  <address@hidden>
 
        testsuite: avoid bogus hostname match from inner test logs.
diff --git a/NEWS b/NEWS
index fc5730f..00fadc3 100644
--- a/NEWS
+++ b/NEWS
@@ -1,5 +1,10 @@
 GNU Autoconf NEWS - User visible changes.
 
+* Major changes in Autoconf 2.64? (????-??-??) [beta]
+
+** The following documented autotest macros are new:
+   AT_CHECK_EUNIT
+
 * Major changes in Autoconf 2.64 (2009-07-26) [stable]
   Released by Eric Blake, based on git versions 2.63b.*.
 
diff --git a/doc/autoconf.texi b/doc/autoconf.texi
index 4f046c0..fa6e1cb 100644
--- a/doc/autoconf.texi
+++ b/doc/autoconf.texi
@@ -22520,6 +22520,8 @@ of debugging scripts has the purpose of easing the 
chase for bugs.
 
 @item all the files created with @code{AT_DATA}
 
address@hidden all the Erlang source code files created with 
@code{AT_CHECK_EUNIT}
+
 @item a log of the run, named @file{testsuite.log}
 @end itemize
 
@@ -22894,6 +22896,54 @@ parameter, and the standard error contents are 
compared with
 @end table
 @end defmac
 
address@hidden AT_CHECK_EUNIT (@var{module}, @var{test-spec}, @ovar{erlflags}, @
+  @ovar{run-if-fail}, @ovar{run-if-pass})
address@hidden
+Initialize and execute an Erlang module named @var{module} that performs
+tests following the @var{test-spec} EUnit test specification.
address@hidden must be a valid EUnit test specification, as defined in
+the @uref{http://@/erlang.org/@/doc/@/apps/@/eunit/@/index.html, EUnit
+Reference Manual}.  @var{erlflags} are optional command-line options
+passed to the Erlang interpreter to execute the test Erlang module.
+Typically, @var{erlflags} defines at least the paths to directories
+containing the compiled Erlang modules under test, as @samp{-pa path1
+path2 ...}.
+
+For example, the unittests associated with Erlang module @samp{testme},
+which compiled code is in subdirectory @file{src}, can be performed
+with:
+
address@hidden
+AT_CHECK_EUNIT([testme_testsuite], address@hidden, address@hidden,
+               [-pa "address@hidden@}/src"])
address@hidden example
+
+This macro must be invoked in between @code{AT_SETUP} and @code{AT_CLEANUP}.
+
+If @var{test-spec} is an invalid EUnit test specification, the test
+group fails.  Otherwise, if the EUnit test passes, shell commands
address@hidden are executed or, if the EUnit test fails, shell
+commands @var{run-if-fail} are executed and the test group fails.
+
+The @code{ERL}, @code{ERLC}, and (optionally) @code{ERLCFLAGS} variables
+must be defined as the path of the Erlang interpreter, the path of the
+Erlang compiler, and the command-line flags to pass to the compiler,
+respectively.  Those variables should be configured in
address@hidden, typically using the @command{AC_ERLANG_PATH_ERL}
+and @command{AC_ERLANG_PATH_ERLC} macros in @file{configure.ac}, and the
+configured values of those variables are automatically defined in the
+testsuite.
+
+Only the generated test Erlang module is automatically compiled and
+executed.  If @var{test-spec} involves testing other Erlang modules,
+e.g. module @samp{testme} in the example above, those modules must be
+compiled explicitly.
+
+If the testsuite is run in verbose mode, with option @option{--verbose},
+EUnit is also run in verbose mode to output more details about
+individual unittests.
address@hidden defmac
+
 
 @node testsuite Invocation
 @section Running @command{testsuite} Scripts
diff --git a/lib/autoconf/autotest.m4 b/lib/autoconf/autotest.m4
index 72b4b7f..405e5cc 100644
--- a/lib/autoconf/autotest.m4
+++ b/lib/autoconf/autotest.m4
@@ -84,6 +84,14 @@ at_top_builddir=\$at_top_build_prefix
 AUTOTEST_PATH='m4_default([$2], [$1])'
 
 SHELL=\${CONFIG_SHELL-'$SHELL'}
+
+# Required to run EUnit unittests.
+ERL='$ERL'
+ERLC='$ERLC'
+ERLCFLAGS='$ERLCFLAGS'
 ATEOF
-])
+],
+[ERL="$ERL"
+ERLC="$ERLC"
+ERLCFLAGS="$ERLCFLAGS"])
 ])# AC_CONFIG_TESTDIR
diff --git a/lib/autotest/Makefile.am b/lib/autotest/Makefile.am
index d310e46..4335549 100644
--- a/lib/autotest/Makefile.am
+++ b/lib/autotest/Makefile.am
@@ -16,7 +16,7 @@
 # along with this program.  If not, see <http://www.gnu.org/licenses/>.
 
 autotestlibdir = $(pkgdatadir)/autotest
-dist_autotestlib_DATA = autotest.m4 general.m4
+dist_autotestlib_DATA = autotest.m4 general.m4 erlang.m4
 nodist_autotestlib_DATA = autotest.m4f
 CLEANFILES = $(nodist_autotestlib_DATA)
 
diff --git a/lib/autotest/autotest.m4 b/lib/autotest/autotest.m4
index dead4c0..f9b8e51 100644
--- a/lib/autotest/autotest.m4
+++ b/lib/autotest/autotest.m4
@@ -45,3 +45,4 @@
 # to the GPL from your modified version.
 
 m4_include([autotest/general.m4])
+m4_include([autotest/erlang.m4])
diff --git a/lib/autotest/erlang.m4 b/lib/autotest/erlang.m4
new file mode 100644
index 0000000..2a1f06a
--- /dev/null
+++ b/lib/autotest/erlang.m4
@@ -0,0 +1,91 @@
+# This file is part of Autoconf.                          -*- Autoconf -*-
+# M4 macros used in running Erlang EUnit unittests in test suites.
+m4_define([_AT_COPYRIGHT_YEARS],
+[Copyright (C) 2009 Free Software Foundation, Inc.])
+
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write to the Free Software
+# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA
+# 02110-1301, USA.
+
+# As a special exception, the Free Software Foundation gives unlimited
+# permission to copy, distribute and modify the configure scripts that
+# are the output of Autoconf.  You need not follow the terms of the GNU
+# General Public License when using or distributing such scripts, even
+# though portions of the text of Autoconf appear in them.  The GNU
+# General Public License (GPL) does govern all other use of the material
+# that constitutes the Autoconf program.
+#
+# Certain portions of the Autoconf source text are designed to be copied
+# (in certain cases, depending on the input) into the output of
+# Autoconf.  We call these the "data" portions.  The rest of the Autoconf
+# source text consists of comments plus executable code that decides which
+# of the data portions to output in any given case.  We call these
+# comments and executable code the "non-data" portions.  Autoconf never
+# copies any of the non-data portions into its output.
+#
+# This special exception to the GPL applies to versions of Autoconf
+# released by the Free Software Foundation.  When you make and
+# distribute a modified version of Autoconf, you may extend this special
+# exception to the GPL to apply to your modified version as well, *unless*
+# your modified version has the potential to copy into its output some
+# of the text that was the non-data portion of the version that you started
+# with.  (In other words, unless your change moves or copies text from
+# the non-data portions to the data portions.)  If your modification has
+# such potential, you must delete any notice of this special exception
+# to the GPL from your modified version.
+#
+# Written by Romain Lenglet
+
+
+## ------------------------ ##
+## Erlang EUnit unittests.  ##
+## ------------------------ ##
+
+# AT_CHECK_EUNIT(MODULE, SPEC, [ERLFLAGS], [RUN-IF-FAIL], [RUN-IF-PASS])
+# ----------------------------------------------------------------------
+# Check that the EUnit test specification SPEC passes. The ERLFLAGS
+# optional flags are passed to the Erlang interpreter command line to
+# execute the test. The test is executed from an automatically
+# generated Erlang module named MODULE. Each call to this macro should
+# have a distinct MODULE name within each test group, to ease
+# debugging.
+# An Erlang/OTP version which contains the eunit library must be
+# installed, in order to execute this macro in a test suite.  The ERL,
+# ERLC, and ERLCFLAGS variables must be defined in configure.ac,
+# typically by using the AC_ERLANG_PATH_ERL and AC_ERLANG_PATH_ERLC
+# Autoconf macros.
+_AT_DEFINE_SETUP([AT_CHECK_EUNIT],
+[## A wrapper to EUnit, to exit the Erlang VM with the right exit code:
+AT_DATA([$1.erl],
+[[-module($1).
+-export([test/0, test/1]).
+test() -> test([]).
+test(Options) ->
+  TestSpec = $2,
+  ReturnValue = case eunit:test(TestSpec, Options) of
+    ok -> 0;
+    _ -> 1
+  end,
+  init:stop(ReturnValue).
+]])
+AT_CHECK(["$ERLC" $ERLCFLAGS -b beam $1.erl])
+## Make EUnit verbose when testsuite is verbose:
+if test -z "$at_verbose"; then
+  at_eunit_options="verbose"
+else
+  at_eunit_options=""
+fi
+AT_CHECK(["$ERL" $3 -s $1 test $at_eunit_options -noshell], [0], [ignore], [],
+         [$4], [$5])
+])
diff --git a/tests/autotest.at b/tests/autotest.at
index 7836439..aa8269f 100644
--- a/tests/autotest.at
+++ b/tests/autotest.at
@@ -1429,3 +1429,77 @@ m4_include([sub/two  spaces.at])
 AT_CHECK([$CONFIG_SHELL ./suite], [0], [stdout])
 AT_CHECK([grep 'two spaces' suite.log], [1])
 AT_CLEANUP
+
+
+## ------------------------ ##
+## Erlang EUnit unittests.  ##
+## ------------------------ ##
+
+AT_SETUP([Erlang Eunit unittests])
+AT_KEYWORDS([Erlang])
+
+mkdir pkg
+mkdir pkg/s pkg/t
+AT_DATA([pkg/configure.ac], [[AC_INIT
+AC_ERLANG_PATH_ERL([not found])
+AC_ERLANG_PATH_ERLC([not found])
+if test "$ERL" = "not found" || test "$ERLC" = "not found"; then exit 77; fi
+AC_ERLANG_CHECK_LIB([eunit], [], [exit 77])
+AC_CONFIG_TESTDIR([t])
+AC_CONFIG_FILES([s/compile])
+AC_OUTPUT
+]])
+cp "$abs_top_srcdir/build-aux/install-sh" pkg
+
+# Erlang module to test:
+AT_DATA([pkg/s/testme.erl],
+[[-module(testme).
+-export([foo/1]).
+foo(1) -> one;
+foo(2) -> two;
+foo(_) -> other.
+]])
+# Corresponding Eunit unittest module:
+AT_DATA([pkg/s/testme_tests.erl],
+[[-module(testme_tests).
+-include_lib("eunit/include/eunit.hrl").
+foo_one_test() -> ?assertEqual(one, testme:foo(1)).
+foo_two_test() -> ?assertEqual(two, testme:foo(2)).
+foo_other_test() -> ?assertEqual(other, testme:foo(42)).
+]])
+# Compilation script:
+AT_DATA([pkg/s/compile.in],
+[["@ERLC@" -b beam testme.erl testme_tests.erl
+]])
+
+cd pkg
+AT_CHECK_AUTOCONF
+cd ..
+
+
+AT_CHECK_AT_PREP([suite],
+[[AT_INIT([suite to check Eunit integration])
+AT_SETUP([my only test])
+AT_CHECK([test -n "$ERL" && test -n "$ERLC"])
+AT_CHECK_EUNIT([my_testsuite], [{module, testme}],
+              [-pa "${abs_top_builddir}/s"])
+AT_CLEANUP
+]], [], [], [], [pkg/t])
+
+cd pkg
+AT_CHECK_CONFIGURE
+cd ..
+
+AT_CHECK([grep '^ERL='\''.*'\' pkg/t/atconfig], [], [ignore])
+AT_CHECK([grep '^ERLC='\''.*'\' pkg/t/atconfig], [], [ignore])
+AT_CHECK([grep '^ERLCFLAGS='\''.*'\' pkg/t/atconfig], [], [ignore])
+
+cd pkg/s
+AT_CHECK([/bin/sh ./compile])
+cd ../..
+
+cd pkg/t
+AT_CHECK([./suite], [0], [ignore])
+cd ../..
+
+AT_CLEANUP
-- 
1.6.3.1