Re: [lmi] Valyuta performance degredation?

[Greg Chicares]

On 1/9/21 12:06 AM, Vadim Zeitlin wrote:
> On Fri, 8 Jan 2021 23:05:42 +0000 Greg Chicares <gchicares@sbcglobal.net> 
> wrote:
[...]
> GC> (I present it this way because it's very easy to paste into a
> GC> spreadsheet for graphing.)
> 
>  Out of curiosity, does your spreadsheet program support pasting
> space-separated values? I had to replace spaces with tabs

Yes, gnumeric accepted the space-separated data. I pasted them
with Shift-Ins, and clicked the "Finish" dialog button immediately.
But perhaps this worked because gnumeric has discerned that I
usually want this behavior, and silently adapted itself.

> GC> This line:
> GC>   git checkout -- Speed_gcc_x86_64-pc-linux-gnu
> GC> prevents:
> GC> 
> GC> ...switching to commit cf62b4482:
> GC> error: Your local changes to the following files would be overwritten by 
> checkout:
> GC>         Speed_gcc_x86_64-pc-linux-gnu
> GC> Please commit your changes or stash them before you switch branches.
> 
>  To be honest I don't really understand why do we need to have a git branch
> for this file. I'd just put it in ".."

But then it would be evanescent, like object files, and only
the latest version would be available.

I want to keep timing data in version control so that I can
see how speed changed across commits simply by using git-log.
That's especially important for "valyuta" branches, where
commits often affect speed. And it's important to compare
such branches against 'master'.

OTOH, these 'Speed*' files are metadata, and their contents
change so much each time that diffs are useless and git just
treats them as binary, so perhaps they should be stored in
some other way. Recently I studied this discussion:

https://stackoverflow.com/questions/12470029/how-to-commit-no-change-and-new-message/12470056#12470056

and considered some alternatives:
 - standalone '--allow-empty' commit messages
 - tags
 - git-notes
instead of files. Of course, that would give up the ability
to use git-log to show a series of differences.

> and avoid switching branches
> unnecessarily.

But aren't frequent switches among branches the hallmark of
sophisticated, mature git usage?

> GC> This line:
> GC>   git switch valyuta/004
> GC> works around this problem:
> GC> 
> GC> /opt/lmi/src/lmi[0]$git switch -
> GC> fatal: a branch is expected, got commit 
> 'd4e029146c6a385820b8952deeabc82558541603'
> GC> 
> GC> although I had thought 'git switch -' should just work.
> 
>  I think "-" only works for switching to a previous branch, but not when
> the previous HEAD was detached.

Thanks, that's exactly the information I needed.

> GC> This line:
> GC>   make clean
> GC> avoid tragic outcomes like:
> GC> 
> GC> /opt/lmi/src/lmi[0]$make cli_timing
> GC> vector::reserve
> GC> 
> GC> /opt/lmi/src/lmi[0]$make cli_timing                     
> GC> Segmentation fault
> GC> 
> GC> I thought that was impossible, because 'git switch' seems to
> GC> 'touch' each file it changes, so that 'make' should just work;
> GC> but that's not what I observed. What am I missing?
> 
>  I think you've replied to your question just below: make doesn't handle
> traveling back in time neither (I used to have a meta-make tool called
> tmake 20+ years ago but I've missed the opportunity to call it tachyon
> make, if only I knew I'd need this feature now...).

Thanks, I thought I must have been doing something wrong.

>  The simplest fix would be to just add --reverse to the git-log command
> above, I don't see any reason to prefer going back in time and I'd even say
> that the plots would look more natural if we didn't do it.

Okay, then the best way would be to start with the oldest commit
(which is what I did) and move forward from there, but to
'make clean' only once, for efficiency (because we take one
large step back, then many small steps forward, and 'make' works
well when we move forward through history).

> GC> This line:
> GC>   #  rm /opt/lmi/gcc_x86_64-pc-linux-gnu/build/ship/*.d
> GC> attempted to work around another problem that I hadn't
> GC> expected to see. I traced it to the autodependency '.d'
> GC> files in /opt/lmi/gcc*/build/ship/ , which don't seem to
> GC> travel back in time very well.
> 
>  This should be fixed by going forward in time too, I believe.

Then the rule is always 'make clean' when we 'git switch --detach'
to an earlier state.

>  Reading this email finally gives me an opportunity to ask you a question I
> wanted to ask since quite some time, looking at your performance
> explorations: do you think it could make sense to collect such kind of
> statistics automatically? Running the script above is probably not that
> much trouble,

Generally, I make small, frequent commits, but do this:
  $time ./nychthemeral_test.sh
(which generates these "Speed*" files) at least before every push.
It may take a while, e.g.:
  ./nychthemeral_test.sh  13956.88s user 740.00s system 1209% cpu 20:15.00 total
but rapid commits are cheap and any errors in them can be
vaporized with 'git rebase --interactive', whereas infrequent
pushes should involve twenty minutes of deliberation because
pushes are forever. At least this way works well for me.

> but wouldn't it be nice if we already had all these numbers?
> As we're now running CI builds on GitHub anyhow, it should be pretty simple
> to build "cli_timing" target in them too and store the result somewhere.
> Then we could have something like the graphs on this page
> 
>       https://rhysd.github.io/github-action-benchmark/dev/bench/
> 
> for lmi too. Would this be useful to have?

Yes. In all of lmi's history, going back to 1998 or so (including
its "ihs" predecessor), we've only had to reissue a monthly
release once IIRC, and that was due to a grave performance
regression--so there's another reason to track speed.