Re: [bug-libsigsegv] Stack overflow handlers with custom stacks

[Marc Nieper-Wißkirchen]

Am Sa., 19. Jan. 2019 um 08:21 Uhr schrieb Marc Nieper-Wißkirchen
<address@hidden>:
>
> Hi,
>
> the following is my use case and I'd like to hear how this is best
> implemented with libsigsegv:
>
> I have a posix thread, whose stack is set with pthread_attr_setstack
> so that I have complete control over where the stack resides in
> memory. At the end of the stack (actually at both ends so that I do
> not have to worry about the stack direction), I have a guard area,
> which is write-protected using mprotect.
>
> Now, want I want is the following: When the thread's stack overflows
> into the guard area, a signal handler shall be called. This signal
> handler shall remove the write-protection, set a flag and return to
> the code that caused the stack overflow.
>
> The thread's main code polls the flag regularly so that it can handle
> the case when the flag is set and only a few pages of stack memory are
> left.
>
> These have been my thoughts on implementing this with libsigsegv:
>
> A) Install a local SIGSEGV handler in the running thread. This makes
> sense because I exactly know the memory area that is being protected.
> However, this does not seem to work because I need an alternative
> stack (à la sigaltstack) for the signal handler as the stack will
> still be write-protected when the handler is entered. But libsigsegv
> does not allow me to set the ON_STACK flag for local handlers.
>
> B) Install a local SIGSEGV handler in another thread. This bypasses
> the problem of the missing alternative stack of idea A. However, this
> seems to overcomplicate the program.
>
> C) Install a stack overflow handler in the running thread. This
> bypasses the problem of A as libsigsegv installs an alternative stack
> in this case. This idea has two drawbacks: First of all, libsigsegv's
> stack overflow handlers go to great lengths to check whether the
> SIGSEGV has come from a stack overflow, which seems unnecessary in my
> case as I exactly know where the stack is. Secondly, can I return from
> a stack overflow handler the same way I can return from a local
> SIGSEGV handler so that the thread's main code is resumed after the
> write protection has been disabled?

I missed the fact that I cannot install per-thread signal handlers
making a lot of the above not meaningful.

What I did, in the end, was to install a dummy stack overflow handler
so that libsigsegv sets the ON_STACK flag and to install a local
SIGSEGV handler to actualy handle the stack overflow.

[...]