Re: [lmi] Continuing deboostification with removing dependency on Boost.Regex

[Vadim Zeitlin]

On Sun, 30 May 2021 18:04:16 +0000 Greg Chicares <gchicares@sbcglobal.net> 
wrote:

GC> GNU parallel would let us run 500 jobs with as much parallelism
GC> as our hardware allows; but wouldn't it still launch 500 processes?
GC> and wouldn't one multithreaded process be faster?

 GNU parallel has "-m" option which can be used (in combination with other
ones) to run just K instances of the program, spreading all the 500
arguments among them. This is still not optimal, as some jobs might finish
earlier than others, but it shouldn't be that far from it.

On Sun, 30 May 2021 20:12:11 +0000 Greg Chicares <gchicares@sbcglobal.net> 
wrote:

GC> On 5/30/21 11:43 AM, Greg Chicares wrote:
GC> > [...] couldn't we parallelize that with a patch like [untested]...
GC> 
GC> Tested patch below.

 Thanks for the numbers!

GC> First, 32-bit msw (wine):
GC> 
GC> ...without patch, for reference:
GC> 
GC> /opt/lmi/src/lmi[0]$time make $coefficiency check_concinnity >/dev/null
GC> make $coefficiency check_concinnity > /dev/null  10.10s user 2.30s system 
299% cpu 4.136 total
GC> 
GC> ...and with the patch below [skip to "Summarized"]:
GC> 
GC> /opt/lmi/src/lmi[0]$rm .concinnity/*                                   
GC> zsh: sure you want to delete more than 100 files in 
/opt/lmi/src/lmi/.concinnity [yn]? y
GC> 
GC> /opt/lmi/src/lmi[0]$time make $coefficiency check_concinnity >/dev/null
GC> make $coefficiency check_concinnity > /dev/null  33.51s user 36.86s system 
97% cpu 1:12.35 total
GC> 
GC> /opt/lmi/src/lmi[0]$time make $coefficiency check_concinnity >/dev/null
GC> make $coefficiency check_concinnity > /dev/null  7.28s user 2.26s system 
768% cpu 1.242 total
GC> 
GC> And here's 64-bit GNU/Linux (after 'rm .concinnity/*' as above):
GC> 
GC> /opt/lmi/src/lmi[0]$time make $coefficiency check_concinnity >/dev/null
GC> make $coefficiency check_concinnity > /dev/null  9.81s user 2.40s system 
338% cpu 3.608 total
GC> /opt/lmi/src/lmi[0]$time make $coefficiency check_concinnity >/dev/null
GC> make $coefficiency check_concinnity > /dev/null  14.75s user 3.62s system 
188% cpu 9.719 total
GC> /opt/lmi/src/lmi[0]$time make $coefficiency check_concinnity >/dev/null
GC> make $coefficiency check_concinnity > /dev/null  7.39s user 2.34s system 
875% cpu 1.111 total
GC> 
GC> Summarized timings, with no changes in tested files so that the
GC> last 'with patch' numbers show how long it takes to do nothing:
GC> 
GC>                  linux    msw
GC>   without patch  4.136   4.136
GC>   with patch     9.719  72.35   empty .concinnity/ directory
GC>   with patch     1.111   1.242  fully populated .concinnity/

 I think there is a typo in the first row, Linux baseline is 3.608
according to the measurement above, not the same as for MSW.

GC> Middle row: the 'wine' startup cost is measurably overwhelming.
GC> But nobody ever really needs to do this with 'wine'.

 The effect of the patch on Linux version is not negligible neither, IMO.

GC> The new '.concinnity/' directory is a bother. With boost::regex,
GC> I don't like this change: I'd rather spend three or four extra
GC> seconds than go to the trouble of setting up and maintaining an
GC> extra directory, and of polishing this patch. I'd view this
GC> sentinel technique as a last resort, which merits further
GC> consideration only if we can't match boost::regex's speed without
GC> using boost (but we anticipate that compile-time regexes will
GC> obviate that).

 I'll try to test this a.s.a.p. but realistically it won't happen until
Tuesday at the earliest, sorry...

VZ

pgpjfWqupVcas.pgp
Description: PGP signature