Re: ANN & GNU Make usage feedback: Qake, featureful build system based on GNU Make 4.0

[Michael Stahl]

On 03.11.2014 19:08, Michael Pankov wrote:
> Full feature list of Qake:
> 
> 1. Compact syntax for definition of entire program build;
> 2. Prevents lengthy meaningless rebuilds - prunes it as early as
> possible. Changing the Git branch doesn't cause the full rebuild
> anymore!;
> 3. Tracks build commands - changes of build system itself are taken into
> account;
> 4. Terse, but useful, output - short command descriptions are output
> instead of full commands by default. At the same time, errors are
> colorized to be easy to notice and provide the full used command when
> user needs it the most;
> 5. Extensible to other types of built entities. Supporting builds of
> static and dynamic libraries with dependencies between them is possible,
> as well as adding ways to build some custom artifacts;
> 6. Tracks header dependencies (of course);
> 7. Uses GNU Make 4.0 internally and exposes it when needed - you can
> still pass custom flags to Make itself for debugging or any other
> purpose.

we have a non-recursive GNU make based build system in LibreOffice too,
heavily based on eval, basically implementing everything in your list
except 3. but in a very project-specific way of course.

> For more information, installation instructions and everything else,
> head to GitHub: https://github.com/mkpankov/qake.
> 
> As for feedback:
> 1. I use $(eval ...) to internalize my generated build description.
> There were two main problems with this for me. 
> 1.1. The first and foremost is that when an error occurs, GNU Make
> reports filename and line number of the call site - that is, of line
> where $(eval $(call ...)) occurs. But what would be infinitely more
> useful is to provide line number relative to beginning of generated
> sequence - that would refer to number of line in generation function
> body.

yes that is a bit frustrating ...

> 1.2. The code that gets evaluated can be viewed by replacing $(eval ...)
> by $(info ...). But, when there are some embedded function calls, like
> (for example) `A := $$(patsubst %/a,%/b,$(SRC))`, it gets only output as
> the function call (not the evaluated values). I know I can print the
> Make database and find the A's value there, but it would be
> significantly more convenient if there would be a way to enforce second
> expansion of such expression. It is even more desirable seeing that
> adding simple $$(info $$(A)) after the definition inside the $(eval ...)
> does output the fully expanded value -- meaning it's already implemented
> in some way.

Norbert Thiebaud has implemented some useful debugging options:

--debug=c and --debug=e to trace $(eval ...) and $(call ...)

https://gerrit.libreoffice.org/gitweb?p=gnu-make-lo.git;a=commit;h=c64dadce9180206f6f8ff5e4c75c091c659ec307

this one allows to find typos when using $(call ...): "support for the
options --warn-undefined-functions"

https://gerrit.libreoffice.org/gitweb?p=gnu-make-lo.git;a=commit;h=fbfa486fcb6251cf700ebd555683279e6f18613f

> 3. During the implementation of features 2. and 3. I discovered it would
> be highly useful if one could specify "proxy" timestamp files instead of
> files that a target really depends on. 
> 
> What I do now is like
> 
> b: a.did_update | a
> 
> and then I specify how to determine if `a` got updated - I hash `a` and
> compare it against previously saved hash value. Then I do `touch
> a.did_update` in case it was really updated, so that `b` would get
> rebuilt too. This seems to be general enough, and would be more easy to
> do, would there be a Make primitive to define a non-timestamp dependency
> of one file on another. As for syntax, I'm not sure I'm able to propose
> it right now. But it's clear that currently you have to do a lot of
> $(patsubst ...) to get a path of proxy file from path of real target and
> then translate it back to path to target to compute, say, a hash of it.
> I'd really like to be able to just mention the real target, what
> "attributes" of it should be tracked, and be done with it. I think new
> automatic variables would be useful.

i'm not sure such a content tracking feature in full generality is
actually useful... for example in our build system we do something like
this with dynamic libraries: when they are rebuilt, the exported symbols
are compared with the previous build and dependent link targets are only
re-linked when the exported symbols change, which eliminates a lot of
time consuming unnecessary linking, and for generated header files,
where one rule generates 4k output files from 4k input files.  but for
most other kinds of targets it is so fast to rebuild them that this sort
of checking doesn't look like it actually saves much time over just
re-building it.

regards,
 michael