Re: [bug-libsigsegv] Stack overflow handlers with custom stacks

[Bruno Haible]

Hi,

Prof. Marc Nieper-Wißkirchen wrote:
> A) Install a local SIGSEGV handler in the running thread. This makes
> sense because I exactly know the memory area that is being protected.
> However, this does not seem to work because I need an alternative
> stack (à la sigaltstack) for the signal handler as the stack will
> still be write-protected when the handler is entered.

Yes, in case of stack overflow any signal handler (SIGSEGV or SIGBUS)
needs to operate on an alternative stack.

> But libsigsegv
> does not allow me to set the ON_STACK flag for local handlers.

You should be able to set the SA_ONSTACK flag yourself:
  1. Install a SIGSEGV / SIGBUS handler through libsigsegv.
  2. Query the installed SIGSEGV signal handler through a call to sigaction().
     Then install the same signal handler again, this time with the
     SA_ONSTACK flag included, through a second call to sigaction().
  3. Likewise for SIGBUS.

> B) Install a local SIGSEGV handler in another thread. This bypasses
> the problem of the missing alternative stack of idea A. However, this
> seems to overcomplicate the program.

This is complicated indeed: You have to avoid that the signal gets
delivered to the thread that is overflowing its stack. To achieve this,
you have to use pthread_sigmask to block the SIGSEGV signals from the
threads.

> C) Install a stack overflow handler in the running thread. This
> bypasses the problem of A as libsigsegv installs an alternative stack
> in this case. This idea has two drawbacks: First of all, libsigsegv's
> stack overflow handlers go to great lengths to check whether the
> SIGSEGV has come from a stack overflow, which seems unnecessary in my
> case as I exactly know where the stack is.

This is a desirable feature for libsigsegv indeed: allow determining
a stack overflow also in other threads than the main thread. Thanks
for mentioning it!

> Secondly, can I return from
> a stack overflow handler the same way I can return from a local
> SIGSEGV handler so that the thread's main code is resumed after the
> write protection has been disabled?

This may work on some platforms, through unportable code that
executes a longjmp() to a jmp_buf (or setcontext() to an ucontext_t)
that has been constructed from the 'ucontext_t' or 'struct sigcontext'
that was passed to the signal handler.

Just returning from the SIGSEGV handler is the more portable appraoch.
It has a problem, though: If your mprotect() call fails to achieve
the purpose, the thread is stuck in an endless loop of SIGSEGVs, which
makes the process 1. eat 100% CPU time and 2. it hard to kill.

> The following enhancement proposals for libsigsegv are related to the above:
> 
> 1) Allow alternative stacks for global and local SIGSEGV handlers.

Alternative stacks are only useful for catching stack overflows.
Therefore the feature I'd prefer to put into libsigsegv is stack overflow
handling for all threads.

> 2) Implement a way to communicate the exact location of the stack in
> memory in case it is known to the program.

Yes, somehow libsigsegv would need to know about the per-thread stacks
as well.

> 3) Export a macro that indicates the direction of growth of the stacks.

It depends on the CPU only. You could use an Autoconf macro like
libsigsegv/m4/stack-direction.m4.

Beware that ia64 is special regarding stack handling, because it has
two stacks, growing in opposite directions.

Bruno