RE: GNU make CPU utilization in cloud environments

[Tamás Fülöp]

Hello everyone,

I have played with the "-j" switch, but it did not affect the performance. I have tried with "-j 2", "-j" and without "-j" options, none of these improved the performance. I have also tried to set the priority level of the make.exe to "High" and "RealTime", but the CPU utilization still sits under 1%. Both 3.81 and 4.1.1 version of GNU make were tried and I am experiencing the same results with both versions.

The original bare make command which I have been using:

make -j -f Makefile 

This command utilizes at least 80% of the CPU on a local machine, but not in the cloud environment.

Reworking the make files is not an option, because we have a tons of them. :(

Thank you for the quick response!

Best regards,

Tamas Fulop

> Subject: Re: GNU make CPU utilization in cloud environments
> From: address@hidden
> To: address@hidden; address@hidden
> Date: Wed, 17 Feb 2016 08:25:55 -0500
>
> On Wed, 2016-02-17 at 10:35 +0100, Tamás Fülöp wrote:
> > I am experiencing problems with the GNU make 3.81 in Amazon-like cloud
> > environment. The "make.exe" does not utilizes the virtual CPUs, the
> > utilization sits around 1% and that makes the building process really
> > slow.
>
> You haven't given us enough information to respond. First, are you
> using the "-jN" flag (where N is some value related to the number of CPU
> cores you have)? Are you using any other flags with your make?
>
> Second, is your makefile environment recursive (with make recipes
> invoking sub-makes), or is it non-recursive (one make process loads all
> the makefiles)? If it's recursive, are you sure you're using the
> $(MAKE) variable to start your sub-makes?
>
>
> In versions of GNU make prior to 4.0, the "job server" feature of GNU
> make was not supported on Windows. This means that if your makefile is
> very recursive you will get very little parallelism in those versions
> regardless of how large your value is for "-j".
>
> If you want more parallelism you'll have to either (a) upgrade to a
> newer version of GNU make, or (b) rework your makefiles so they're non
> -recursive.