Re: GMAKE 3.81 vs GMAKE 4.2

[nikhil jain]

This is what NFS specs says about it -

... to recover from an ESTALE error, an application must close the file or
directory where the error occurred, and reopen it so the NFS client can
resolve the pathname again and retrieve the new file handle.

And that is what safe_stat is doing - opening the file which results in
resolving pathname.

On Tue, Dec 17, 2019 at 1:17 PM nikhil jain <address@hidden> wrote:

> I will reply on this sooner with code snippets.
>
> Just to again confirm that this actually solved a very critical issue in
> my project,
>
> A rule was processed and then as soon as it ends, the next dependent rule
> processed. It has to work on the object file generated by the first rule.
> But the timestamp of the file was older so the next rule did not process. I
> am on NFS. As I mentioned earlier also, looks like NFS is not updated on
> the submission host from where I am calling make command. But when i use
> safe_stat, it seems the rules are working fine and while make is doing a
> dependency graph for the next rule, it finds the object file from previous
> rule with newer timestamp and process the current rule.
>
> I assume that this is because we are doing an open and read operation in
> safe_stat which might remove the stale old file handle and make can see a
> newer updated file.
>
> Also, the comment mentioned in the misc.c in the code you can see it has
> to be related with NFS caching -
>
>     /* For safety's sake, read something to make sure NFS attribute cache
>  *      * is updated
>  *           */
>
> To answer your first question -
>
> YES, it works if i do a make clean (which deletes all the object files)
> and the next make command will work. But for incremental build, It wont
> work. I can reproduce the issue 8 out of 10 times with stat and 0 out of
> 1000 times with safe_stat..
>
> Cant share exact code snippets as I am not allowed to.. but this is what
> is happening...
>
>
> On Tue, Dec 17, 2019 at 2:19 AM Paul Smith <address@hidden> wrote:
>
>> On Wed, 2019-12-04 at 18:14 +0530, nikhil jain wrote:
>> > Just for information.
>> >
>> > Issue is not reproduced when replacing stat with safe_stat() which
>> > double checks the file for stale handle.
>> >
>> > On Wed, Dec 4, 2019 at 12:51 PM nikhil jain <address@hidden>
>> > wrote:
>> > > A rule which executes in host A creates a .o file.
>> > > Once it is done, make checks for the next rule which is dependent
>> > > on this one.
>> > >
>> > > But on the submission host while making the dependency graph again,
>> > > it sees that the .o timestamp is not newer which means it is still
>> > > reading the old .o file.
>> > >
>> > > So it doesn't build the target.
>> > > I know this is normal as it is on NFS so latency is there.
>>
>> Can it work if you just delete the .o file before you invoke the
>> command to recreate it?  Then it will either not exist, or it will
>> exist and have the right timestamp.
>>
>> > > But can I do something from make side to cover this latency? I
>> > > tried putting sleep. It works but we don't know what is the optimal
>> > > sleep value.
>> > >
>> > > Also, sleep slows down the complete build time.
>> > >
>> > > I see a function safe_stat() and now it is convert d to just stat()
>> > >
>> > > What was the difference actually ? Why we moved from safe_stat() to
>> > > stat?
>>
>> That change happened long before GNU make 3.81 was released.  In fact,
>> it was made by Roland before I even started maintaining GNU make; from
>> the ChangeLog:
>>
>> Mon Dec 11 22:26:15 1995  Roland McGrath  <address@hidden>
>>   ...
>>         * misc.c (safe_stat): Function removed.
>>         * read.c, commands.c, remake.c, vpath.c: Use plain stat instead of
>>         safe_stat.
>>
>> There is no information in the ChangeLog or commit message as to why
>> this was done.
>>
>> This change would have been included in every GNU make starting with
>> version 3.75 which was released in August 1996.
>>
>> However, looking back into the history I can see that all that the
>> safe_stat() function did was loop if we received an EINTR error.  The
>> current code already does this; stat() calls are embedded in the
>> EINTRLOOP macro which does the same thing.
>>
>> I also can't see how looping on EINTR would help your problem, because
>> you only get EINTR if the stat() was interrupted by a signal and that
>> wouldn't help your setup.
>>
>> Perhaps if you explained (a) what your replacement safe_stat() function
>> did, (b) where you found the safe_stat you're using, and (c) what
>> changes you made to GNU make code to call safe_state, some of the
>> mystery could be removed.
>>
>>