[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: How do parallel builds scale?
From: |
Howard Chu |
Subject: |
Re: How do parallel builds scale? |
Date: |
Thu, 03 Mar 2011 08:25:58 -0800 |
User-agent: |
Mozilla/5.0 (X11; Linux x86_64; rv:2.0b11pre) Gecko/20110130 Firefox/4.0b11pre SeaMonkey/2.1b2pre |
Ludovic Courtès wrote:
Hello,
There was recently discussion as to how configure could be parallelized.
I ran a series of build time measurements on a 32-core machine, with
make -jX, with X in [1..32], and the results are available at:
It's been my experience that compile-oriented builds tend to be I/O bound and
generally have excess CPU capacity in the range of 30-40%. As such I always
use make -jX where X is number of cores * 1.5, which tends to keep the idle
CPU percentage below 5%.
http://hubble.gforge.inria.fr/parallel-builds.html
There are also insightful per-package details:
http://hubble.gforge.inria.fr/parallel-build-details.html
There are packages whose configuration phase is noticeably longer than
the build time.
Yes. I've seen several projects where the autotool files are larger than the
actual program source code. IMO, any time your build overhead takes up more
than 1% of your total project size, you've done something wrong.
OTOH, packages with a long build time (with a sequential time in
boldface) don’t scale well either, but that is not configure’s fault.
If only more people would learn to write proper Makefile dependencies...
http://highlandsun.com/hyc/#Make
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
- How do parallel builds scale?, Ludovic Courtès, 2011/03/03
- Re: How do parallel builds scale?,
Howard Chu <=
- Re: How do parallel builds scale?, Ralf Wildenhues, 2011/03/04
- Re: How do parallel builds scale?, Bob Friesenhahn, 2011/03/04
- Re: How do parallel builds scale?, Ludovic Courtès, 2011/03/04
- Re: How do parallel builds scale?, Ralf Wildenhues, 2011/03/04
- Re: How do parallel builds scale?, Ludovic Courtès, 2011/03/07
- Re: How do parallel builds scale?, Ralf Wildenhues, 2011/03/07
- Re: How do parallel builds scale?, Bob Friesenhahn, 2011/03/07
- Re: How do parallel builds scale?, Ludovic Courtès, 2011/03/08