[bug #18396] stack size setrlimit call interacts badly with Solaris/x86 kernel bug

[Scott McPeak]

URL:
  <http://savannah.gnu.org/bugs/?18396>

                 Summary: stack size setrlimit call interacts badly with
Solaris/x86 kernel bug
                 Project: make
            Submitted by: smcpeak
            Submitted on: Tuesday 11/28/2006 at 20:16
                Severity: 3 - Normal
              Item Group: Bug
                  Status: None
                 Privacy: Public
             Assigned to: None
             Open/Closed: Open
         Discussion Lock: Any
       Component Version: 3.81
        Operating System: POSIX-Based
           Fixed Release: None

    _______________________________________________________

Details:

I recently experienced the following mysterious phenomenon:

* Running a program from the shell, it crashes (it dereferences NULL
on purpose; that is all it does).

* Invoking that same program from with a Makefile, it does not crash.
Instead, it happily reads/writes page 0.

After considerable investigation, I have discovered that the cause is
two interacting issues:

* There is a kernel bug in Solaris/x86 that causes page 0 to be mapped
if the stack limit is 'unlimited'; see

  http://bugs.opensolaris.org/bugdatabase/view_bug.do?bug_id=6374692
  
* GNU make was modified on 2001-09-06 (between 3.79.1 and 3.80) by
Paul Eggert to "Get rid of any avoidable limit on stack size" (quote
from ChangeLog).  See the 'setrlimit' call in main.c.

Consequently, whether a program is running under 'make' greatly
affects how it behaves, as the child processes inherit the resource
limits as well.

While the kernel issue is clearly a bug, I think 'make' behavior is a
misfeature as well.  Generally one expects resource limits to not be
silently changed by shells and shell-like programs such as 'make'.
That 'make' does this is troubling; among other things, diagnosing the
consequences is difficult (I investigated many other possible causes
before finding it).  The Solaris kernel bug is just one way such a
silent change might be manifested.

If 'make' needs to allocate a large amount (megabytes) of data, it
would be better to do so on the heap, both from a portability
standpoint (the stack size) and from a performance standpoint (it
messes up the normally good locality of stack access).

Alternatively, if it must allocate on the stack, then detecting and
complaining about a too-low limit would be better in my opinion than
silently changing it.  It's easy to uncap the stack size explicitly
in build scripts and whatnot when truly necessary.

Output of uname -a:

  SunOS tainted.sf.coverity.com 5.10 Generic_118844-26 i86pc i386 i86pc
Solaris

Output of make -v:

  GNU Make 3.81
  Copyright (C) 2006  Free Software Foundation, Inc.
  This is free software; see the source for copying conditions.
  There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A
  PARTICULAR PURPOSE.

  This program built for i386-pc-solaris2.10





    _______________________________________________________

Reply to this item at:

  <http://savannah.gnu.org/bugs/?18396>

_______________________________________________
  Message sent via/by Savannah
  http://savannah.gnu.org/