[Mingw-cross-env-list] Framework for testing (was: GCC 4.4.1)

[Volker Grabsch]

Tony Theodore <address@hidden> schrieb:
> I've been trying to find
> some small applications to build as a test and I realised, after looking at
> the IMCROSS project, that samples and demos are the ideal place to start.

On the one hand, that's true. On the other hand, we have to make
a difference between what I'd call "build tests" and what I'd call
"run tests".

For build tests, it is important that they build without any linker
errors. For run tests, it is important that they run on Wine and are
completely non-interactive.

Most samples and demos are only usable as build tests. However, these
are the ones that we maybe should generate automatically, e.g. from
the library files (*.a), rather than using those provided by the
scripts.

However, reusing the tests from the original sources might also be
a clever idea, especially for unit tests.

> At
> the moment I just changed the qt configure options, but I'd like to hear
> some opinions on how best to structure the building of them to keep them
> separate from the rest of the libraries and toolchain.

I think we should be able to build them completely outside the
scope of their packages. This would allow us to test the *_config
and pkg_config scripts, too. For instance, a test program "test_$PKG.c"
for a normal library "$PKG" should compile via something like:

$(TARGET)-gcc `$(TARGET)-pkg-config --libs --cflags $PKG` -o test_$PKG 
test_$PKG.c

without having to add any additional libraries or compiler options.

> I imagine a new make target, but beyond that, I'm not sure:
> 
> Should dependencies be built?

Yes, they should. When testing a library that is not built yet,
or which is not up-to-date, the library should be built before
the test.

> Where to put the makefiles and binaries?

It would be great if we could handle all testing within a single
(and simple) Makefile, maybe even in the main Makefile.

We might have to add some testing hints directly to the corresponding
src/*.mk files, but I hope this won't be necessary.

> What to call it (samples/tests)?

The Make target should be:

    make check

That's the accepted standard for running test suites. We would
also have to add separate "check-$PKG" targets, such that running
the tests is not an all-or-nothing thing.


Greets,

    Volker

-- 
Volker Grabsch
---<<(())>>---
Administrator
NotJustHosting GbR