Re: Some experience with the igc branch

[Gerd Möllmann]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Gerd Möllmann <gerd.moellmann@gmail.com>
>> Cc: pipcet@protonmail.com,  ofv@wanadoo.es,  emacs-devel@gnu.org,
>>   eller.helmut@gmail.com,  acorallo@gnu.org
>> Date: Tue, 24 Dec 2024 15:12:40 +0100
>> 
>> Eli Zaretskii <eliz@gnu.org> writes:
>> 
>> > Now, with that definition, isn't specpdl stack part of "Lisp data"?
>> > If so, and if we can safely access it from a signal handler, why do we
>> > need to move it aside at all?  And how would the "message handler" be
>> > different in that aspect from a signal hanlder?
>> 
>> We're coming from the problem that MPS uses signals for memory barriers.
>> On platforms != macOS. And I am proposing a solution for that.
>> 
>> The SIGPROF handler does two things: (1) get the current backtrace,
>> which does not trip on memory barriers, and (2) build a summary, i.e.
>> count same backtraces using a hash table. (2) trips on memory barriers.
>
> Can you elaborate on (2) and why it trips?  I guess I'm missing
> something because I don't understand which code in record_backtrace
> does trip on memory barriers and why.

Ok, (2) begins as shown below.

  static void
  record_backtrace (struct profiler_log *plog, EMACS_INT count)
  {
    log_t *log = plog->log;
    get_backtrace (log->trace, log->depth);
  --- (2) begins after this line -------------------------------
    EMACS_UINT hash = trace_hash (log->trace, log->depth);

The SIGPROF can have interrupted Emacs at any point, both the MPS thread
and all others. MPS may have been doing arbitrary stuff when
interrupted, and Emacs threads too. Memory barriers may be on
unpredictable segments of memory, as they usually are, as part of MPS'
GC implementation. Do you agree with this picture?

Elsewhere I tried to explain why I think this works up to the line
marked (2) above. Now enter trace_hash. Current implementation:

  static EMACS_UINT
  trace_hash (Lisp_Object *trace, int depth)
  {
    EMACS_UINT hash = 0;
    for (int i = 0; i < depth; i++)
      {
        Lisp_Object f = trace[i];
        EMACS_UINT hash1;
  #ifdef HAVE_MPS
        hash1 = (CLOSUREP (f) ? igc_hash (AREF (f, CLOSURE_CODE)) : igc_hash 
(f));
                 ^^^^^^^^       ^^^^^^^^  ^^^^

The constructs I marked with ^^^ all access the memory of F. F is a
vectorlike, it's memory is managed by MPS in an MPS pool that uses
memory barriers, so the memory of F can currently be behind a barrier.
It doesn't have to, but it can.

When we access F's memory and it is behind a barrier, the result is a
nested SIgSEGV while handling SIGPROF.

More code accessing memory that is potentially behind a barrier follows
in record_backtrace.