Re: Shrinking the C core

[Gerd Möllmann]

Arthur Miller <arthur.miller@live.com> writes:

> Gerd Möllmann <gerd.moellmann@gmail.com> writes:
>> I'm pretty sure that CL packages could be added to Emacs as it is, if
>> some people would work on it.
>
> With "CL packages" you mean namespaces? There seem to be already a
> branch that implements them, but I don't know how well it works, I
> haven't tried it.

Yup, that's mine, as a pastime :-).  I have no plans with that.

>> I'm also pretty sure that an incremental + generational GC could be
>> added, at least as an option, because I would have almost done it some
>> 20+ years ago.  It was torpedoed by a patent issue concerning
>> mostly-copying GC.  The patent has since expired. A lot of work, of
>> course.  I think some people do or have done something in this area, but
>> I don't know details.
>
> I am not very familiar wth GC:s implementation more then just some
> bired-eye overview. SBCL recently started to move towards non-moving
> GC to help with the speed, notably when calling native code which does
> not like it's pointers moved underneath, but I am not expert on details
> there, these what I have got from the paper:
>
> https://applied-langua.ge/~hayley/swcl-gc.pdf

Interesting, thanks for the pointer!

>> I'm not at all sure that non-cooperative multi-threading could be added
>> to Emacs.  But I'm also not sure how a CL core would help here.
>
> They are exposing posix threads and have done some work to make at least
> parts of the Lisp system work well with threads, and it seems it is
> working well for many applicaitons.
>
> http://www.lichteblau.com/sbcl/doc/manual/sbcl/Implementation-_0028Linux-x86_0029.html#Implementation-_0028Linux-x86_0029

Yes, I remember to some degree.  I think Daniel Barlow started adding
thread support to SBCL at the time I constributed to CMUCL.  ISTR some
communiaction with him about the implemenattion of dynamic bindings in
the presence of threads.

> I think there is also a missconception in Emacs community that Emacs
> loop itself has to be parallelized; I am not sure it is needed;

Don't know what the "loop" refers to.

I think the biggest problem with uncooperative multi-threading in Emacs
is that Emacs has so much global state.  As a consequence, it's unsafe
to let two threads use anything in the C code in parallel.  Maybe one
could think of using some Python-like GIL, but that's kind of pointless,
isn't it?

> I think
> for many people it would be enough to expose threading in form of "js
> workers" or something like that. It can be done with processes of
> course, but people seem to constantly scream about it in disucssions.

Parallel worker threads have the same problem as above.  And cooperative
threads are there already, I think.

> CL has things like lparallel and green threads built on top of
> hardware threads, so even there is a bit of job already done. I am
> sure all that can be done in Emacs too, but I think, both communities
> would be more helped if we perhaps used sbcl and interested
> individuals helped make sbcl runtime better intead of reduplication
> the entire effort.

Is this realistic?  I mean how many people would be interested to do that?

>> On the other hand, I'm pretty convinced that an Emacs core written in CL
>> would have to be close to 100% compatible with the existing C core to be
>> accepted by users.  That includes a CL rewrite of the C Elisp, including
>> byte code interpreter.
>
> Yes, my conclusion too.
>
>> accepted by users.  That includes a CL rewrite of the C Elisp, including
>> byte code interpreter.
>
> I am not sure how much of byte-interpretter is needed; I was thinking
> how byte interpretter and native compiler fitt there. Oviously since
> sbcl is a compiler, with don't need all that stuff, but I am not sure
> how much of byte code intepretter is needed. I am sure we need to
> understand all of the syntax, since byte code is a valid elisp,
> according to the manual; so the reader have to be able to read the
> syntax I guess, as printed representation, and has to print same stuff
> back to feed into elisp functions.

Not sure what you are saying.  Something has to execute the bytecode,
or not?

>> That's a massive endeavor.  My hair stands up when I remember the
>> compatibility problems I faced with the new redisplay ages ago.
>> Multiply that by some factor > 1.  But maybe that's a burnt child
>> dreading the fire :-).
>
> Yes, I know. I am fully aware that it is an impossibility for someone
> alone, even for a very few. I don't think it is a burnt child, since
> yes, the character renderer of Emacs has to be implemented if Emacs
> applications will run unchanged.

(I think you misunderstoof the phrase.  I'm the burnt child...).

> Hopefully it will be possible to implement Emacs stuff as a special kind
> of terminal/character renderer over some sort of tree/graph structure. I
> think CLOS and CL have much better tools to refactor that stuff than C,
> but what do I know, I haven't tried that and I am not sure if I will
> tbh. I am bolling with the ideas. I have seen what they do in other
> similar CL software (Hemlock, McClIM, Lem), perhaps there is something
> that can be reused there, but I don't know how much and what yet. And,
> yes text properties are a special chapter on its own :).

So, your plan would be to re-implement redisplay in CL.  Good look with
that.  And a ton of other stuff, like the stuff Eli mentioned.  And the
result of the whole massive effort is then 100% compatible with current
Emacs.

That's what I call quatsch, sorry, but honestly.

Work on Lem if you like CL that much :-)