Re: An Emacs benchmarking suite

[Phillip Lord]

Eli Zaretskii <address@hidden> writes:
>> Even so, how would one account for external factors in the operating
>> system?  Perhaps when the test is performed, the deltas between
>> commits are given as percentages instead of CPU time.  Once we have
>> that, it would be great to have tests run multiple times and/or on
>> various devices to refine the data further.
>
> Performance should indeed compare several versions n the same system.


I wonder if there is a way to work out a base line value. Benchmarks
could then look for multiples of this.

So, something like time to create a big list (for CPU), and time to read
a defined file (for IO). Then you could say "this file should parse in
1x IO base-line + 10x CPU base-line.

That way the bench marks could become part of the test set. If things
got significantly slower, tests would fail.

Phil