Re: GMAKE 3.81 vs GMAKE 4.2

[nikhil jain]

I will reply on this sooner with code snippets.

Just to again confirm that this actually solved a very critical issue in my
project,

A rule was processed and then as soon as it ends, the next dependent rule
processed. It has to work on the object file generated by the first rule.
But the timestamp of the file was older so the next rule did not process. I
am on NFS. As I mentioned earlier also, looks like NFS is not updated on
the submission host from where I am calling make command. But when i use
safe_stat, it seems the rules are working fine and while make is doing a
dependency graph for the next rule, it finds the object file from previous
rule with newer timestamp and process the current rule.

I assume that this is because we are doing an open and read operation in
safe_stat which might remove the stale old file handle and make can see a
newer updated file.

Also, the comment mentioned in the misc.c in the code you can see it has to
be related with NFS caching -

    /* For safety's sake, read something to make sure NFS attribute cache
 *      * is updated
 *           */

To answer your first question -

YES, it works if i do a make clean (which deletes all the object files) and
the next make command will work. But for incremental build, It wont work. I
can reproduce the issue 8 out of 10 times with stat and 0 out of 1000 times
with safe_stat..

Cant share exact code snippets as I am not allowed to.. but this is what is
happening...


On Tue, Dec 17, 2019 at 2:19 AM Paul Smith <address@hidden> wrote:

> On Wed, 2019-12-04 at 18:14 +0530, nikhil jain wrote:
> > Just for information.
> >
> > Issue is not reproduced when replacing stat with safe_stat() which
> > double checks the file for stale handle.
> >
> > On Wed, Dec 4, 2019 at 12:51 PM nikhil jain <address@hidden>
> > wrote:
> > > A rule which executes in host A creates a .o file.
> > > Once it is done, make checks for the next rule which is dependent
> > > on this one.
> > >
> > > But on the submission host while making the dependency graph again,
> > > it sees that the .o timestamp is not newer which means it is still
> > > reading the old .o file.
> > >
> > > So it doesn't build the target.
> > > I know this is normal as it is on NFS so latency is there.
>
> Can it work if you just delete the .o file before you invoke the
> command to recreate it?  Then it will either not exist, or it will
> exist and have the right timestamp.
>
> > > But can I do something from make side to cover this latency? I
> > > tried putting sleep. It works but we don't know what is the optimal
> > > sleep value.
> > >
> > > Also, sleep slows down the complete build time.
> > >
> > > I see a function safe_stat() and now it is convert d to just stat()
> > >
> > > What was the difference actually ? Why we moved from safe_stat() to
> > > stat?
>
> That change happened long before GNU make 3.81 was released.  In fact,
> it was made by Roland before I even started maintaining GNU make; from
> the ChangeLog:
>
> Mon Dec 11 22:26:15 1995  Roland McGrath  <address@hidden>
>   ...
>         * misc.c (safe_stat): Function removed.
>         * read.c, commands.c, remake.c, vpath.c: Use plain stat instead of
>         safe_stat.
>
> There is no information in the ChangeLog or commit message as to why
> this was done.
>
> This change would have been included in every GNU make starting with
> version 3.75 which was released in August 1996.
>
> However, looking back into the history I can see that all that the
> safe_stat() function did was loop if we received an EINTR error.  The
> current code already does this; stat() calls are embedded in the
> EINTRLOOP macro which does the same thing.
>
> I also can't see how looping on EINTR would help your problem, because
> you only get EINTR if the stat() was interrupted by a signal and that
> wouldn't help your setup.
>
> Perhaps if you explained (a) what your replacement safe_stat() function
> did, (b) where you found the safe_stat you're using, and (c) what
> changes you made to GNU make code to call safe_state, some of the
> mystery could be removed.
>
>