Re: [lmi] Continuing deboostification with removing dependency on Boost.Regex

[Greg Chicares]

On 5/30/21 10:48 PM, Vadim Zeitlin wrote:
> On Sun, 30 May 2021 18:04:16 +0000 Greg Chicares <gchicares@sbcglobal.net> 
> wrote:
[...]
> GC> /opt/lmi/src/lmi[0]$time make $coefficiency check_concinnity >/dev/null
> GC> make $coefficiency check_concinnity > /dev/null  9.81s user 2.40s system 
> 338% cpu 3.608 total
> GC> /opt/lmi/src/lmi[0]$time make $coefficiency check_concinnity >/dev/null
> GC> make $coefficiency check_concinnity > /dev/null  14.75s user 3.62s system 
> 188% cpu 9.719 total
> GC> /opt/lmi/src/lmi[0]$time make $coefficiency check_concinnity >/dev/null
> GC> make $coefficiency check_concinnity > /dev/null  7.39s user 2.34s system 
> 875% cpu 1.111 total
> GC> 
> GC> Summarized timings, with no changes in tested files so that the
> GC> last 'with patch' numbers show how long it takes to do nothing:
> GC> 
> GC>                  linux    msw
> GC>   without patch  4.136   4.136
                       ^^^^^
             should be 3.608
> GC>   with patch     9.719  72.35   empty .concinnity/ directory
> GC>   with patch     1.111   1.242  fully populated .concinnity/
> 
>  I think there is a typo in the first row, Linux baseline is 3.608
> according to the measurement above, not the same as for MSW.

Thanks. Corrected above for the record.

> GC> Middle row: the 'wine' startup cost is measurably overwhelming.
> GC> But nobody ever really needs to do this with 'wine'.
> 
>  The effect of the patch on Linux version is not negligible neither, IMO.

Yes, with 826 files in the src/ directory, the incremental cost
to run test_coding_rules against them all is
 - three seconds, with one process that loops over them serially, but
 - nine seconds, with a separate process for each, in parallel,
making serial faster than parallel with --jobs=32: that is, the
overhead for 'make' to start all those processes is quite large.

Of course, when all 826 sentinels exist and are up to date, all
that work is saved. But I had hoped that 826 parallel processes
would beat one serial process even if no sentinels exist, so that
this would be a pure win in all cases. Furthermore...

> GC> The new '.concinnity/' directory is a bother. With boost::regex,
> GC> I don't like this change: I'd rather spend three or four extra
> GC> seconds than go to the trouble of setting up and maintaining an
> GC> extra directory, and of polishing this patch.
"Polishing" this 'make'-with-sentinels method would mean
 - modifying 'test_coding_rules' to print nothing when it
   detects no error;
 - perhaps simplifying it to expect only one filename
   argument--but then 'test_coding_rules_test.sh' would
   need to be modified, too; and
 - creating '.concinnity' whenever necessary, and removing
   it in 'make clean' (over time, the directory could become
   cluttered with old files).
We'd also have to think through what happens when git
commands like 'revert', 'mv', and 'switch' are run (I don't
see any potential problems there, but it's the problems I
don't see that are a concern).

OTOH, always testing all files is simple and robust.