Re: [lmi] Remarkable performance problem

[Greg Chicares]

[...replying to the message that started the thread in the hope of
preserving some semblance of threading--I haven't received copies
of my own more recent messages from the list server...]

On 2018-02-28 01:54, Greg Chicares wrote:
[...]
> Today Kim sent me a census file that runs quickly enough for her, but
> shockingly slowly for me.

There would seem to be something about that file that causes
difficulties with 'wine'. The reported symptoms are greatly
mitigated by either of the following changes:

(1) Simply save the file, then close and reload lmi, and load
the saved file.

(2) Use the new paste-from capability, if you can penetrate
the wall of undiscoverability behind which it's hidden, to
produce a TSV file from the original census; then paste it
back into (an empty census in) lmi, and run that.

The original input file is eccentric in some ways:

- It's CRLF-delimited ASCII according to the 'file' command,
although my copy of LibreOffice seems to think it's Turkish,
so maybe there's an exotic character in it somewhere. However,
I had previously stripped all CR characters from a copy of
the original file, and observed no improvement in symptoms.
And this command:
  grep -P -n '[^\x00-\x7F]' ../../input/clean1691.cns
finds no non-ASCII characters, so the Turkish thing is
probably just a LibreOffice configuration oddity.

- It's also an obsolescent version of the xml layout, because
it came from an outdated third-party vendor system; loading
it may cause multiple_cell_document::cell_sorter() to be run,
and that step would be skipped (as being unnecessary) for
(1) and (2) above.

Anyway, even if we figure out why modifications (1) and (2)
above treat the worst symptom (making the GUI so unresponsive
that simply closing lmi takes longer than XFCE expects, so it
pops up an "lmi has stopped responding" messagebox), another
issue remains: performance deteriorates during a census run.
I.e., "Census | Run case" pops up a progress dialog which,
after the first hundred cells or so are processed speedily,
estimates how long the command will take to complete, but
that estimate grows and grows. With either (1) or (2), with
an empty $WINEDEBUG, an early estimate of about 65 seconds
grows to about 360 or 380 by the end. I haven't yet ruled
out the possibility that the later cells are more difficult
to process, but that seems unlikely; I suspect that running
the first 100 or 200 cells depletes some scarce resource.