Re: Benchmark tracking

[Neil Jerram]

address@hidden (Ludovic Courtès) writes:

> Hi Neil,
>
> Neil Jerram <address@hidden> writes:
>
>>   http://ossau.homelinux.net/~neil/
>
> This looks really good to me.  The only thing that’s missing is a
> mapping from units on the X axis to commits.

That's at http://ossau.homelinux.net/~neil/bm_master_x.html, and it
is linked from the main page above.

> Looking at http://ossau.homelinux.net/~neil/bm_master_i.html, it seems
> that several things got slightly slower around x = 25.

x = 25 => early morning of 2009-10-23, commit
04c68c039194f33d5bd7e8b1f21eba7c8bd6adbe.

I agree that many tests made a bit of a jump then, but remember that
that could be caused just by something unusual happening on my build
machine.  I'm not aware of anything unusual on that date... but it's
still possible.

Also, in many cases the loss seems to have been recovered again since
then.

And, looking at the commits on 2009-10-22:

        Compile Guile modules with `-Wunbound-variable'.

        Fix typos leading to unbound variable references.

        Adjust `unbound-variable' GOOPS heuristic for `goops... 

        Fix bytecode disassembler.

        SRFI-88: Call `read-set!' at compile time and run time.

None of these look like they could have caused a slow down.

I'm not sure we can rely very much on these graphs for day-to-day
changes, because I haven't eliminated all possible external factors, so
there is considerable random variation.  I'd say their value is more to
give us confidence that we don't slow Guile down over the medium term.

Regards,
      Neil