[bug-libsigsegv] Stack overflow handlers with custom stacks

[Marc Nieper-Wißkirchen]

Hi,

the following is my use case and I'd like to hear how this is best
implemented with libsigsegv:

I have a posix thread, whose stack is set with pthread_attr_setstack
so that I have complete control over where the stack resides in
memory. At the end of the stack (actually at both ends so that I do
not have to worry about the stack direction), I have a guard area,
which is write-protected using mprotect.

Now, want I want is the following: When the thread's stack overflows
into the guard area, a signal handler shall be called. This signal
handler shall remove the write-protection, set a flag and return to
the code that caused the stack overflow.

The thread's main code polls the flag regularly so that it can handle
the case when the flag is set and only a few pages of stack memory are
left.

These have been my thoughts on implementing this with libsigsegv:

A) Install a local SIGSEGV handler in the running thread. This makes
sense because I exactly know the memory area that is being protected.
However, this does not seem to work because I need an alternative
stack (à la sigaltstack) for the signal handler as the stack will
still be write-protected when the handler is entered. But libsigsegv
does not allow me to set the ON_STACK flag for local handlers.

B) Install a local SIGSEGV handler in another thread. This bypasses
the problem of the missing alternative stack of idea A. However, this
seems to overcomplicate the program.

C) Install a stack overflow handler in the running thread. This
bypasses the problem of A as libsigsegv installs an alternative stack
in this case. This idea has two drawbacks: First of all, libsigsegv's
stack overflow handlers go to great lengths to check whether the
SIGSEGV has come from a stack overflow, which seems unnecessary in my
case as I exactly know where the stack is. Secondly, can I return from
a stack overflow handler the same way I can return from a local
SIGSEGV handler so that the thread's main code is resumed after the
write protection has been disabled?

The following enhancement proposals for libsigsegv are related to the above:

1) Allow alternative stacks for global and local SIGSEGV handlers.
2) Implement a way to communicate the exact location of the stack in
memory in case it is known to the program.
3) Export a macro that indicates the direction of growth of the stacks.

Thanks a lot,

-- Marc