Re: Emacs repository benchmark: bzr and git (rerun)

[Teemu Likonen]

Teemu Likonen kirjoitti:

> I did some benchmarking in git and bzr repositories of Emacs. Some
> numbers: 89711 revisions (by "git log --pretty=oneline | wc -l"),
> 2825 files. Both repositories seem to have just linear history
> converted from CVS repo. Both have the same head revision which is
> 481c2a1e31f32c8aa0fb6d504575b75a18537788 (git) and
> revid:cvs-1:tsdh-20080318180244-lxbzttdnh6ecqbka (bzr).
>
> Repositories/branches are pulled from here:
> git: git://git.sv.gnu.org/emacs.git
> bzr: http://bzr.notengoamigos.org/emacs/trunk/
>
> My system is AMD Sempron 3000+ with 2 GB memory and it's running
> Debian GNU/Linux 4.0. I'm using the latest development versions of
> both git (1.5.5.rc0.6.gdeda) and bzr (1.4dev). I just measured with
> 'time' command how long it takes to run certain commands.


Hey! I realised that the emacs bzr repository was not fully optimized 
with "bzr pack" command. By running this command it really improves the 
performance of bzr; now I'm getting similar numbers as some of you. I 
really apologise for the misleading information I have spread. I want 
to correct my mistakes by running my tests again (see below).

This raises questions though. I had downloaded the premade emacs bzr 
repo from <http://bzr.notengoamigos.org/> and it seems to have its repo 
pretty much optimized since it performs much better than my previous 
benchmark. I had done almost nothing with the repository after that, 
just some bzr-pulls and performance tests. How come the emacs bzr 
repository slows down so much and so quickly? My experience is that you 
definitely want to run "bzr pack" quite often.

Anyway, here are the same tests with bzr-packed and git-gc'd 
repositories. This test shows that both have improved their 
performance, especially bzr. I did run all the tests three times to see 
if caches have effect. They didn't: all the three runs gave very much 
the same numbers. I picked up the best one anyway.

Again, I'm really sorry about my previous tests.


> Viewing history
> ---------------

I want to point out that in git you can always see the log 
_immediately_, no matter how long it takes to display the whole thing. 
In bzr these numbers reflect pretty much the time to get anything 
visible at all after entering the command.


> The complete history:
>
> $ time git log >/dev/null
> real    0m5.741s
>
> $ time bzr log >/dev/null
> real    3m15.708s

git: 0m3.348s
bzr: 1m15.143s


> Last 100 revisions:
>
> $ time git log -100 >/dev/null
> real    0m0.011s
>
> $ time bzr log -l100 >/dev/null
> real    2m10.270s

git: 0m0.009s
bzr: 0m26.562s


> Last 10 revisions:
>
> $ time git log -10 >/dev/null
> real    0m0.007s
>
> $ time bzr log -l10 >/dev/null
> real    2m9.163s

git: 0m0.005s
bzr: 0m25.519s


> The complete history of a single file:
>
> $ time git log src/keymap.c >/dev/null
> real    0m9.240s
> (The same as above but with detecting and following possible
> renames:) $ time git log --follow src/keymap.c >/dev/null
> real    0m17.891s
>
> $ time bzr log src/keymap.c >/dev/null
> real    3m35.431s

git: 0m5.127s
git: 0m8.953s (with --follow)
bzr: 0m55.461s


> Differences between revisions
> -----------------------------

> View changes introduced in given revision:
>
> (This shows also the commit message, author and date.)
> $ time git show 2635714f3dac5f24eb1997cbf97285810f6799c0 >/dev/null
> real    0m0.012s
>
> $ time bzr diff -c revid:cvs-1:wohler-20080318101724-c3ofm3vslli3wfwl
> >/dev/null real    2m40.467s

git: 0m0.010s
bzr: 0m23.315s


> Show differences between two revisions:
>
> $ time git diff HEAD~10..HEAD~4 >/dev/null
> real    0m0.076s
>
> $ time bzr diff -r -11..-5 >/dev/null
> real    2m44.214s

git: 0m0.068s
bzr: 0m24.140s


> $ time git diff HEAD~4.. >/dev/null
> real    0m0.072s
>
> $ time bzr diff -r -5.. >/dev/null
> real    1m21.836s

git: 0m0.060s
bzr: 0m12.889s


> Creating a branch
> -----------------
>
> With git I chose "git checkout -b" instead of "git branch" because
> the former also checks out the files as does "bzr branch". The bzr
> branch is created inside the same shared repository so that the
> common objects are shared.
>
>
> Create new topic branch based on the head revision of the main
> development branch:
>
> $ time git checkout -b topic master >/dev/null
> real    0m0.062s
>
> $ time bzr branch trunk topic >/dev/null
> real    0m7.249s

git: 0m0.043s
bzr: 0m4.504s


> Create new topic branch based on earlier revision of main development
> branch:
>
> $ time git checkout -b topic master~4 >/dev/null
> real    0m0.085s
>
> $ time bzr branch -r -5 trunk topic >/dev/null
> real    2m51.551s

git: 0m0.062s
bzr: 0m30.120s


> Compare branches' commit histories
> ----------------------------------
>
> In above benchmark I created branch 'topic' which is based on earlier
> revision of main development branch. In this test I compared commands
> which display commits that are missing from 'topic' branch compared
> to the main development branch (four commits in total).
>
>
> $ time git log topic..master >/dev/null
> real    0m0.006s
>
> $ time bzr missing --theirs-only ../trunk >/dev/null
> real    18m25.173s

git: 0m0.004s
bzr: 13m48.239s


> Annotate/blame a file (src/keymap.c)
> ------------------------------------
>
> $ time git blame src/keymap.c >/dev/null
> real    0m8.753s
>
> $ time bzr blame src/keymap.c >/dev/null
> real    0m58.296s

git: 0m7.954s
bzr: 0m17.114s