Re: [lmi] Cross-linking issues

[Greg Chicares]

On 2016-01-26 15:13, Vadim Zeitlin wrote:
> On Tue, 26 Jan 2016 14:58:56 +0000 Greg Chicares <address@hidden> wrote:
[...]
> GC> But this is puzzling, because the native-built version definitely seems
> GC> to have been built with threads
> ...
> GC> yet 'install_libxml2_libxslt.make' says '--without-threads':
> 
>  The only guess I have is that this particular copy of the DLL had been
> built before the --without-threads option was added in r6482. Otherwise I
> really don't see how could it be possible.

Aha--you're quite correct.

>  Could you please try rebuilding the native libxml2-2.dll

[Wow, that took forever, and not just the 'configure' phase. Even with
'--jobs=16', native compiling is really slow.]

> and look at make
> output to check that --without-threads is really passed to configure?

/opt/lmi/local/bin[0]$ls -l xmllint.exe
-rwxr-xr-x 1 earl None 189918 Jan 26 15:33 xmllint.exe
/opt/lmi/local/bin[0]$./xmllint.exe --version
C:\opt\lmi\local\bin\xmllint.exe: using libxml version 20626
   compiled with: Tree Output Push Reader Patterns Writer SAXv1 FTP HTTP 
DTDValid HTML Legacy C14N Catalog XPath XPointer XInclude ISO8859X Unicode 
Regexps Automata Expr Schemas Debug

Now that's identical to the cross build:

/cygdrive/c/downloads[0]$./xmllint.exe --version
C:\downloads\xmllint.exe: using libxml version 20626
   compiled with: Tree Output Push Reader Patterns Writer SAXv1 FTP HTTP 
DTDValid HTML Legacy C14N Catalog XPath XPointer XInclude ISO8859X Unicode 
Regexps Automata Expr Schemas Debug

And now the acid test.

/cygdrive/c/downloads[0]$cygcheck ./xmllint.exe
C:\downloads\xmllint.exe
  C:\downloads\libxml2-2.dll  <-- CROSS BUILT
    C:\WINDOWS\system32\KERNEL32.dll
      C:\WINDOWS\system32\ntdll.dll
    C:\WINDOWS\system32\msvcrt.dll
    C:\WINDOWS\system32\WS2_32.dll
      C:\WINDOWS\system32\ADVAPI32.dll
        C:\WINDOWS\system32\RPCRT4.dll
          C:\WINDOWS\system32\Secur32.dll
      C:\WINDOWS\system32\WS2HELP.dll
  C:\WINDOWS\system32\USER32.dll
    C:\WINDOWS\system32\GDI32.dll

It still used a local cross-built copy, so let's spoil that:

/cygdrive/c/downloads[0]$mv ./libxml2-2.dll CROSSlibxml2-2.dll
/cygdrive/c/downloads[0]$cygcheck ./xmllint.exe
C:\downloads\xmllint.exe
  C:\opt\lmi\local\bin\libxml2-2.dll  <--NATIVE BUILT
    C:\WINDOWS\system32\KERNEL32.dll
      C:\WINDOWS\system32\ntdll.dll
    C:\WINDOWS\system32\msvcrt.dll
    C:\WINDOWS\system32\WS2_32.dll
      C:\WINDOWS\system32\ADVAPI32.dll
        C:\WINDOWS\system32\RPCRT4.dll
          C:\WINDOWS\system32\Secur32.dll
      C:\WINDOWS\system32\WS2HELP.dll
  C:\WINDOWS\system32\USER32.dll
    C:\WINDOWS\system32\GDI32.dll

And now it works:

/cygdrive/c/downloads[0]$./xmllint.exe --version
C:\downloads\xmllint.exe: using libxml version 20626
   compiled with: Tree Output Push Reader Patterns Writer SAXv1 FTP HTTP 
DTDValid HTML Legacy C14N Catalog XPath XPointer XInclude ISO8859X Unicode 
Regexps Automata Expr Schemas Debug
/cygdrive/c/downloads[0]$

confirming that the native and cross binaries are compatible.

But the really important news is that now a cross-built 'lmi_cli.exe' runs,
and produces valid output. It doesn't yet exactly match the output from a
native-built 'lmi_cli_shared.exe' as I had hoped, but the reason might be
a subtle difference in compiler options.

Here's the result of 'make system_test' with a large number of proprietary
products. We compare every single number calculated (many thousands of
floating-point numbers) for each testdeck. If they all match perfectly,
we say so:
  1478 system-test files compared
  941 system-test files match
and thus:
  537 system-test files differ
Then, for each mismatching testdeck, we calculate the relative error in
each of those thousands of numbers, and choose the error with the largest
absolute value. If that's less than 1e-14, we accept it; else we print
its name and error statistics. Out of those 537, only 101 are listed in
this case [names obscured below], so this is pretty good already. One-cent
differences attributable to rounding
  sample.000000018.test   Summary: max abs diff: 0.01 max rel err:  2.53218e-008
and minuscule differences
  exp_rat.000000001.test   Summary: max abs diff: 4.65661e-010 max rel err:  
4.47558e-014
are probably okay, but something like this
  sample.000000101.test   Summary: max abs diff: 1.97 max rel err:  3.09282e-005
would need to be looked at. Upgrading from gcc-3.4.5 to 4.9.1 produced
test differences with about the same complexion as this run; but this one
compares cross and native builds with 4.9.1 . Maybe everything will match if
I rebuild everything, taking great care to use the same options everywhere.

System test:
6b.000000000.test   Summary: max abs diff: 0.01 max rel err:  6.20283e-010
6b.000000004.test   Summary: max abs diff: 0.01 max rel err:  1.03486e-008
7b.000000000.test   Summary: max abs diff: 0.01 max rel err:  7.27282e-010
7b.000000010.test   Summary: max abs diff: 0.01 max rel err:  1.36893e-008
7b.000000019.test   Summary: max abs diff: 0.01 max rel err:  1.49045e-008
8-12345.000000000.test   Summary: max abs diff: 0.01 max rel err:  1.41412e-009
8-12345.000000004.test   Summary: max abs diff: 0.01 max rel err:  7.89976e-009
8a.000000001.test   Summary: max abs diff: 0.01 max rel err:  4.09679e-008
8b.000000000.test   Summary: max abs diff: 0.32 max rel err:  3.2519e-007
8b.000000002.test   Summary: max abs diff: 0.01 max rel err:  6.5411e-009
8b.000000006.test   Summary: max abs diff: 0.01 max rel err:  9.73695e-009
8b.000000011.test   Summary: max abs diff: 0.32 max rel err:  2.5814e-005
exp_rat.000000000.test   Summary: max abs diff: 9.31323e-010 max rel err:  
4.47558e-014
exp_rat.000000001.test   Summary: max abs diff: 4.65661e-010 max rel err:  
4.47558e-014
exp_rat.000000002.test   Summary: max abs diff: 1.27329e-010 max rel err:  
1.20211e-014
exp_rat12.000000001.test   Summary: max abs diff: 0.01 max rel err:  
3.75294e-007
exp_rat12.000000002.test   Summary: max abs diff: 2.17331e-006 max rel err:  
5.63314e-009
gu1.000000000.test   Summary: max abs diff: 0.01 max rel err:  7.39362e-010
gu1.000000006.test   Summary: max abs diff: 0.01 max rel err:  2.38383e-009
gu1.000000007.test   Summary: max abs diff: 0.01 max rel err:  7.91746e-009
gu1.000000021.test   Summary: max abs diff: 0.01 max rel err:  2.6477e-008
gu2.000000000.test   Summary: max abs diff: 0.01 max rel err:  6.84136e-010
gu2.000000012.test   Summary: max abs diff: 0.01 max rel err:  2.34116e-008
gu2.000000019.test   Summary: max abs diff: 0.01 max rel err:  1.57784e-008
gv1.000000001.test   Summary: max abs diff: 5.82077e-011 max rel err:  
1.17538e-014
gv1.000000002.test   Summary: max abs diff: 7.27596e-012 max rel err:  
1.76524e-014
gv1.000000003.test   Summary: max abs diff: 2.91038e-011 max rel err:  
1.50412e-014
gv1.000000005.test   Summary: max abs diff: 3.63798e-012 max rel err:  
3.30346e-014
gv1.000000007.test   Summary: max abs diff: 1.81899e-012 max rel err:  
1.49103e-014
gv1.000000008.test   Summary: max abs diff: 3.63798e-012 max rel err:  
1.63707e-014
gv1.000000010.test   Summary: max abs diff: 7.27596e-012 max rel err:  
1.60804e-014
gv1.000000013.test   Summary: max abs diff: 9.09495e-013 max rel err:  
1.53299e-014
gv1.000000018.test   Summary: max abs diff: 1.45519e-011 max rel err:  
1.61775e-014
gv1.000000030.test   Summary: max abs diff: 1.45519e-011 max rel err:  
1.39919e-014
gv1.000000031.test   Summary: max abs diff: 1.45519e-011 max rel err:  
1.51545e-013
gv1.000000039.test   Summary: max abs diff: 1.81899e-012 max rel err:  
2.14114e-014
gv1.000000040.test   Summary: max abs diff: 1.45519e-011 max rel err:  
2.13597e-014
gv1.000000041.test   Summary: max abs diff: 1.09139e-011 max rel err:  
8.17579e-014
gv1.000000042.test   Summary: max abs diff: 1.45519e-011 max rel err:  
1.17686e-013
init_min_prem.000000010.test   Summary: max abs diff: 4.65661e-010 max rel err: 
 1.12649e-014
init_min_prem.000000011.test   Summary: max abs diff: 4.65661e-010 max rel err: 
 1.07286e-014
init_min_prem.000000013.test   Summary: max abs diff: 5.82077e-010 max rel err: 
 1.07347e-014
mec8.000000000.test   Summary: max abs diff: 0.01 max rel err:  5.48486e-011
mec8.000000022.test   Summary: max abs diff: 0.01 max rel err:  2.85416e-009
mec9.000000022.test   Summary: max abs diff: 0.01 max rel err:  2.85416e-009
mec9.000000073.test   Summary: max abs diff: 2.91038e-011 max rel err:  
1.67664e-014
mp1.000000000.test   Summary: max abs diff: 0.01 max rel err:  5.60901e-008
mp1.000000001.test   Summary: max abs diff: 0.01 max rel err:  1.75098e-007
reg_d_group.000000000.test   Summary: max abs diff: 2.32831e-010 max rel err:  
1.91025e-013
reg_d_group.000000001.test   Summary: max abs diff: 2.32831e-010 max rel err:  
1.91025e-013
sample.000000000.test   Summary: max abs diff: 3.36 max rel err:  3.70464e-008
sample.000000018.test   Summary: max abs diff: 0.01 max rel err:  2.53218e-008
sample.000000035.test   Summary: max abs diff: 0.01 max rel err:  3.89375e-008
sample.000000036.test   Summary: max abs diff: 0.01 max rel err:  5.70609e-009
sample.000000037.test   Summary: max abs diff: 0.01 max rel err:  5.70609e-009
sample.000000087.test   Summary: max abs diff: 0.01 max rel err:  2.17686e-008
sample.000000088.test   Summary: max abs diff: 0.01 max rel err:  4.33233e-008
sample.000000098.test   Summary: max abs diff: 1.39 max rel err:  3.09282e-005
sample.000000099.test   Summary: max abs diff: 0.02 max rel err:  1.04524e-007
sample.000000101.test   Summary: max abs diff: 1.97 max rel err:  3.09282e-005
sample.000000113.test   Summary: max abs diff: 0.01 max rel err:  8.48968e-009
sample.000000150.test   Summary: max abs diff: 0.01 max rel err:  6.61701e-009
sample.000000153.test   Summary: max abs diff: 0.01 max rel err:  6.61701e-009
sample.000000178.test   Summary: max abs diff: 0.01 max rel err:  2.08814e-009
13bmp.000000000.test   Summary: max abs diff: 4.65661e-010 max rel err:  
4.3785e-014
13bmp.000000001.test   Summary: max abs diff: 5.09317e-011 max rel err:  
2.435e-014
13bmp.000000002.test   Summary: max abs diff: 2.36469e-010 max rel err:  
4.3785e-014
td5.000000000.test   Summary: max abs diff: 0.0400001 max rel err:  3.81811e-011
td5.000000025.test   Summary: max abs diff: 0.01 max rel err:  6.13966e-010
td5.000000027.test   Summary: max abs diff: 0.01 max rel err:  4.44522e-009
td5.000000047.test   Summary: max abs diff: 0.01 max rel err:  1.77781e-009
td5.000000059.test   Summary: max abs diff: 0.01 max rel err:  5.75556e-009
td5.000000060.test   Summary: max abs diff: 0.01 max rel err:  5.75556e-009
td5.000000063.test   Summary: max abs diff: 0.01 max rel err:  5.75556e-009
td5.000000064.test   Summary: max abs diff: 0.01 max rel err:  5.75556e-009
td5.000000089.test   Summary: max abs diff: 0.01 max rel err:  3.41905e-009
td5.000000102.test   Summary: max abs diff: 0.01 max rel err:  3.82569e-009
ten.000000000.test   Summary: max abs diff: 2.07188 max rel err:  5.23904e-008
ten.000000014.test   Summary: max abs diff: 3.63798e-012 max rel err:  
1.92501e-014
ten.000000016.test   Summary: max abs diff: 0.01 max rel err:  7.77613e-009
ten.000000022.test   Summary: max abs diff: 3.63798e-012 max rel err:  
6.73143e-014
ten.000000026.test   Summary: max abs diff: 2.07188 max rel err:  1.21124e-005
ten.000000037.test   Summary: max abs diff: 7.27596e-012 max rel err:  
1.26175e-012
ten.000000038.test   Summary: max abs diff: 3.63798e-012 max rel err:  
1.20275e-014
ten.000000039.test   Summary: max abs diff: 2.91038e-011 max rel err:  
3.20326e-014
ten.000000040.test   Summary: max abs diff: 1.63709e-011 max rel err:  
2.09007e-013
ten.000000041.test   Summary: max abs diff: 7.27596e-012 max rel err:  
6.22965e-014
ten.000000053.test   Summary: max abs diff: 2.32831e-010 max rel err:  
1.23776e-014
ten.000000076.test   Summary: max abs diff: 4.65661e-010 max rel err:  
1.17376e-014
ten.000000085.test   Summary: max abs diff: 1.81899e-012 max rel err:  
5.56384e-014
ten.000000086.test   Summary: max abs diff: 1.81899e-012 max rel err:  
1.73771e-014
ten.000000094.test   Summary: max abs diff: 1.45519e-011 max rel err:  
4.43008e-014
ten.000000095.test   Summary: max abs diff: 2.91038e-011 max rel err:  
4.97996e-014
ten.000000102.test   Summary: max abs diff: 9.31323e-010 max rel err:  
5.58829e-014
ten.000000115.test   Summary: max abs diff: 2.32831e-010 max rel err:  
3.11436e-014
ten.000000121.test   Summary: max abs diff: 1.16415e-010 max rel err:  
1.25053e-013
ten.000000122.test   Summary: max abs diff: 2.91038e-011 max rel err:  
1.30513e-014
ten.000000123.test   Summary: max abs diff: 1.45519e-011 max rel err:  
4.43264e-014
ten.000000124.test   Summary: max abs diff: 1.16415e-010 max rel err:  
2.82559e-014
ten.000000125.test   Summary: max abs diff: 1.45519e-011 max rel err:  
2.01539e-014
ten.000000128.test   Summary: max abs diff: 3.13776e-011 max rel err:  
2.45155e-014
Files /opt/lmi/test/md5sums-20160126T1550Z and /opt/lmi/touchstone/md5sums 
differ
*** System test failed ***
  1478 system-test files compared
  941 system-test files match
  537 system-test files differ
  0 system-test files missing
...system test completed.