[NS] Emacs 28 (IOSurface) renders much slower than Emacs 27 on macOS Big Sur

[Yiming Chen]

Hi there!
I know that we've switched to a IOSurface based rendering for nsterm in
master branch.
Now master renders more precisely than emacs-27 (i.e. less flickers or
blank screens.)
But I feel that its rendering performance is much worse due to the new
buffering mechanism backed by IOSurface.


Several cases that I can feel this worse performance:
1. moving a maximized emacs frame from a larger screen to a smaller
   screen (I use hammerspoon to do that)
   I can see the emacs-28 frame shrink to the smaller screen size, then
   move to the smaller screen
2. scrolling
   I set `scroll-conservatively' to 101 in order to achieve the
   smooth-scrolling effect.
   When I scroll down a large text buffer constantly (hold `j' in
   evil-mode or `C-n' in emacs-mode),
   emacs-28 hangs more times than emacs-27 (emacs-27 scrolls very
   smoothly except it would flicker from time to time)

   See also
   
<https://www.reddit.com/r/emacs/comments/mhdjxb/macos_emacs_with_metal_support/guhbck4/>
   for a previous discussion.
3. child-frame
   child-frame like company-box appears more slowly in emacs-28 than
   emacs-27

Sorry I haven't found a way to measure the rendering performance except
the `scroll-up-benchmark' script from Alan Third in the reddit comment
mentioned above.


I think this poor rendering performance obscures the improvements
brought by native-compile.
So I want to fix this performance issue and make Emacs runs on macOS as
fast as on Linux.

I've spent the last several days trying to understand how nsterm works
and if we can use a more efficient rendering API.
But I failed completely, mostly because I have no Cocoa development
experience before.
Here are the 3 steps I want to take to improve the rendering
performance:
1. Try to restore the nsterm behavior from Emacs27 (which is much
   simpler than master IMHO), and start again from there

   I've reverted the code and made it compile (see
   
<https://github.com/emacs-mirror/emacs/compare/master...dsdshcym:revert-to-emacs-27-nsterm>)
   But Emacs failed to start and raised an error `Font ‘Menlo’ is not
   defined'

2. Use layer-based view for rendering

   I found that macvim also faced the similar issue when macOS 10.14
   came out (<https://github.com/macvim-dev/macvim/issues/751>).
   They used NSImage to implement a double rendering solution
   (<https://github.com/macvim-dev/macvim/pull/757/files>), which was
   also slower than before.
   Then they switched from NSImage to layer-based view in
   
<https://github.com/macvim-dev/macvim/commit/dba6293677e0633917e3054cfddec1293e5ab3fb>.
   So I wonder if we can do the same.

3. Switch to CALayer (Core Animation) to use GPU (Metal) rendering for
   even better performance than before

   If 2 (layer-based view) is possible, then I hope we can leverage
   CALayer to make nsterm renders even faster.


I was so desperate about this issue, so I also tried other window
systems on macOS:
1. emacs-pgtk (<https://github.com/masm11/emacs/issues/95>):
   technically emacs-pgtk should work on macOS with gtk3, but the
   performance was even worse
2. emacs-ng + webrender
   (<https://github.com/emacs-ng/emacs-ng/issues/207>)
   webrender hasn't been working on macOS neither

If anyone can point a direction for me so I can investigate more, that
would be the best.
Thanks in advance!

Cheers,
Yiming