Re: [lmi] Running unit tests in less time

[Greg Chicares]

On 2017-05-03 01:20, Vadim Zeitlin wrote:
[...]
> GC> hardcoding unique names for each unit test's temporary files (instead of
> GC> hardcoding non-unique names as at present). I think that's what I'll do.
> 
>  It's probably justified on the basis of not worrying about a problem which
> is rather unlikely to arise, but I do feel uncomfortably about using hard
> coded file names, even for (or maybe especially for) the temporary files.

I actually did something rather different. The temporary files that caused
filesystem races turned out to be static files that are part of a normal
lmi installation. The only reason for writing them in the build directory
was to avoid the filesystem altogether--i.e., std::ifstream("foo") finds
file 'foo' in the current directory very robustly. To prevent the race
problem (which seemed much worse with x86_64 than with i386), I simply
changed the applicable tests to look in '/opt/lmi/data', which is what
'input_test' already did (we already depend heavily on boost:: or
std::experimental::filesystem, so there's no getting away from it).

You might prefer that $datadir be configurable, but that's a separate
problem. At worst we now have three unit tests that all have the same
single issue, instead of one with that issue and two others with data
races. At best, with an lmi installation in the hardcoded directory,
there are no problems...with lmi's own (non-auto-) make files. As for
autotools, let's consider the issues you discuss here:

  https://github.com/vadz/lmi/issues/53

| 5. path_utility_test fails because it relies on MSW-specific behaviour
| of preventing the open files from being deleted

Opening '.tsv' or '.pdf' output with the msw software usually provided by
corporations causes the files to be locked, and lmi works around this
"feature" only because it must, not because it's locking is desirable.
I suppressed the applicable tests for platforms other than msw.

| 6. system_command_test fails because md5sum --check --status eraseme
| returns 256 instead of expected 1 ... the Unix version ... really ought
| to use WIFEXITED() and WEXITSTATUS()

At least for command-line applications, I'm reluctant to venture beyond
standard C++. lmi's system_command() is just a wrapper around std::system(),
except in the msw case where something more elaborate was required to work
around the "/c: can't open input file: /c" problem that 'system_command.hpp'
describes. The only use case in production is invoking FOP, but we want to
replace FOP; otherwise more elaborate failure analysis would be wanted.

As for the rest...

| 1. cache_file_reads_test fails because it expects to find sample.ill file
| in the current directory

| 2. configurable_settings_test fails because configurable_settings.xml
| doesn't exist

| 3. input_test fails because /opt/lmi/data/single_cell_document.xsd doesn't
| exist

| 4. mc_enum_test fails because Data directory 'xxx' contains no product files

...couldn't all those problems could be solved by installing lmi into
'/opt/lmi/data'?