[lmi] Finish submodules soon? [Was: Use submodules for, and newer versions of, libxml2 and libxslt]

[Greg Chicares]

[...replying now to the most time-sensitive portion only...]

On 2020-10-01 15:36, Vadim Zeitlin wrote:
[...]
>  I plan to make more PRs to switch to using submodules for wx and wxPdfDoc
> too in the near future. I don't know if it's worth doing it for all
> libraries from install_miscellania.make, but I'd like to do it for at least
> xmlwrapp (which, BTW, could/ought to be installed from install_x*.sh now).

Can you estimate how long the wx* and *wrapp things would take
to submodule-ize? Would one week be enough?

> Please let me know if you have any objections.

libxslt had broken (AFAICS, due to a 'wine' upgrade affecting
snprintf() in the msw emulator's C runtime library), and we
had to fix it. Fixing it was an intractable task that was made
tractable by replacing the tarball-plus-funky-makefile silliness
with a modern git-submodule-plus-shell-script reimplementation.

This is not some run-of-the mill change that only requires
re-running 'make'. It's a change to the build system, which needs
to be tested rigorously (and production changes have to be
suspended while that's going on). Each cycle of rigorous testing
costs about the same, so we'd prefer to squash as many such
changes together as possible, and test them all together, OAOO,
and then close the books on submodule-izing for a long time.

Therefore, striking while the iron is hot, this is the perfect
time to submodule-ize the whole lmi universe, including
 - wxWidgets and wxPdfDoc,
 - xmlwrapp and xsltwrapp,
and anything else that should be a submodule. Candidates, from
'install_miscellanea.make':

  xmlwrapp_archive := xmlwrapp-0.9.0.tar.gz

Should become a submodule.

  PETE [not in 'install_miscellanea.make']

Mentioned because its present location, in src/tools/pete-2.1.1,
seems ideal. That subdirectory might as well be a hundred meters
underground in Spitsbergen and encased in kryptonium. Maybe that's
the perfect place for some other artifacts I can't bear to part
with, like my Funkadelic LPs and some of the following?

  boost_archive    := boost_1_33_1.tar.bz2

Once we upgrade lmi to use std::filesystem, we might keep this
old archive around for 'test_coding_rules' only, because its
regex implementation is much faster than libstdc++'s was, last
time we looked. We could extract only as much as we need for
that limited purpose, and put it in a Spitsbergen subdirectory.
Wouldn't a git submodule be overkill, though? We're never going
to pull it from any upstream, or build it as a library.

  cgicc_archive    := cgicc-3.1.4.tar.bz2

Same question as for boost: just move it to Spitsbergen,
because it's just a bunch of source files that we aren't
going to pull from upstream?

  jing_archive     := jing-20091111.zip
  trang_archive    := trang-20091111.zip

Whether it's official or not, this exists:
  https://github.com/relaxng/jing-trang
but do we care? I don't think we'd ever want to build java
stuff from source.

  sample_archive   := lmi-data-20050618T1440Z.tar.bz2

It's just a bag-of-files from what we might nostalgically
think of as our "ftp" area. The question is whether git
is the new ftp (in addition to everything else it does).
I don't know--is it? Aren't there binaries in that archive?
Isn't that like putting a vinyl LP in a CD jewel case?

Putting that all together, I think we should try to migrate
exactly the things you named:
 - wxWidgets and wxPdfDoc
 - xmlwrapp and xsltwrapp
into submodules, and leave all the others alone. Anything
marked for Spitsbergenization can be nibbled at, piecemeal,
whenever we like, because they don't require any outage
on production machines--if cgicc is it
  /opt/lmi/somewhere
today, and tomorrow it's in
  /opt/lmi/src/lmi/test/Funkadelic
and makefiles are updated concurrently with that change,
then production machines can be updated thus:
  make
which is easy and hiccup-less, as opposed to something like
  ./install_msw_make
which takes more effort. How much more? Not much if you
have a server-grade CPU and even a low-end SSD. Much, if
you have a rotary disk and two CPU cores. Maybe next year
that won't matter any more, but right now it does.