Re: Rethinking the design of xwidgets

[Arthur Miller]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>>> I don't think that algorithm is less optimal nowadays than it was back
>>> then; the really interesting question is how much it saves us when
>>> compared to simply redisplaying all those lines.
>
> Right.  The relative performance of different kinds of code has probably
> changed significantly, so it's possible that it's faster to just redraw
> the whole window than to try and figure out which part to redraw (and
> even try and find some scrolling opportunities in there).
> Then again, it's also quite possible that those optimizations are still
> very much worthwhile.

I have never really thought that the graphic performance is bottleneck in
Emacs either. I thinking more in terms of code simplification.

I have read the comment in disp.c and looked a bit into the display
routine at some time just for the fun, and I am impressed and as
currently wouldn't dare to touch anything there :-).

>> In case where there is a discret gfx card (i.e. Nvidia/AMD) it is
>> probably faster to send everything to GPU and ask it to render a 
>> giant texture and then use it as XWindow pixmap, or something similar
>> then to figure out on CPU all the stuff that should not be displayed.
>
> I think it's much harder than it sounds: most of the work needs access
> to data structures that aren't friendly to GPUs, so the work of
> providing data to a GPU in an appropriate form would probably not be
> much more less in most cases than the drawing itself.

As said above, I don't consider myself as well introduced to display
code, so consider this mostly as musings and curiousity: are you sure it
is that hard? What I think of is that display routine has to get this
data from lisp structures to draw to OS window anyway. While it is done,
instead of drawing to the screen, stash it into some GPU friendly
structure, i.e. "render" it into a VBO or something similar that can be
sent to gpu.

Maybe this does not need to be pulled-out; I am not sure about this; but
where and when is rendering state updated? As I understand all colours
and other rendering data for text goes through properties? Couldn't
that data be updated at same time in rendering structure when they are
updated as lisp property? By adding a hook to propertize or whatever
does that and then mark text as dirty? Just wild thoughts, don't take it
as concrete suggestion. Maybe I completely don't understand how it works.

I am not sure about the details how rendering could be done, that is
just a thought about the portion of getting state data for text from
lisp.

> (talking-about-things-one-does-not-understand-mode 1)

Nice markers you have :-)

> But indeed, maybe we could split the "drawing" from the glyph matrices
> to the glass into a first step that draws into a "pixmap matrix" and
> a second step that draws from it onto the glass.  This should make it
> unnecessary to try and "scroll" (parts of) the display since it should
> be "just as fast" to copy from the pixmap matrix as it is to copy from
> the current glass's content.

If you think in terms of "glass" or "layers" would it be possible to do
this:

Create a list of textures (to be used as layers) for a displayed buffer
(temp buffers and maybe some special buffers that are not displayed does
not need it). Render window background into one texture. Render buffer
text into another texture, render cursor into third texture and render
current line into last texture. Renderer could then composit those into
final display.

Then for example if you enabled GL, you could just move a textured
polygon up or down which should be extremely fast. Zoom in zoom out
would be adjusting moving volume (I suppose glOrtho for projection).

Rendering highlight for current line, selections etc would be composing
the texture with a colored polygon and doing some blending operations
possibly in shader to invert rendered text (I think), alternatively
render to some other texture and then compose over the old one; I don't
know just fast thinking.

If you are familiar with Qt, I think that is similar strategy they do
in their Qml. They have a scenegraph and they render each widget's state
(I think) to a texture and then just compose/display those textures at
render time. I am not sure though, it was long time I was reading about
how they do stuff.


By the way, about egl: nice thing with it is that it enables easy render
to texture, and also headless rendering (no X11 required) on gpu, but it
can also be used to render to an OS surface directly (a window).

Consider Emacs image: emacs inserts images as character properties and
display routine is aware of it's dimensions etc. If one could insert
image as a placeholder (say some imaginary format 'egl-image')
one could then render and image as a texture via egl with some lisp
commands; some graphics api not yet existing in elisp that would render
via opengl into egl texture, and then the image buffer, a pixel array as
floats, could be copied back to Emacs and displayed in image
placeholder. I don't know how to hook it up into image routines in
Emacs, was looking at it some time ago, but I didn't found it back then,
and didn't have time to go back. 

That is less efficient since it copies data back and forth between
cpu/gpu, but it would still be much more efficient then going through
external process and file system as Emacs does now. Maybe if gl context
is enable in entire frame, it would be possible to enable directly
render opengl content to the Emacs window portion that image placeholder
is. Some translation of pixel coordinate and other synchronisation would
be needed, but I don't think it is impossible or extremenly hard to do
from the opengl point of view. 

Maybe I don't understand all the pecularities and intricacies, I
myself, certainly don't know how to hook up all this into Emacs nor if
it was even possible to pull of; I am just theorycrafting atm :-).

>
> I think the performance of the redisplay is by and large poorly
> understood.  While there are known cases where people experience "slow
> redisplay" it's usually very unclear what that means concretely.
> In many cases this can be completely unrelated to the actual redisplay
> code (e.g. the time is spent running some expensive code off of
> `post-command-hook` or font-lock or younameit).

You are probably correct about that one.