Re: Emacs Lisp JIT Compiler

[Stefan Monnier]

> Daniel> Likewise, it'd be fantastic to compile regular expressions to DFAs and
> Daniel> then generate machine code for the DFAs. You can't go faster than 
> that.
> I've been meaning to experiment with this using Stefan's lex.el.
> It seems to me that the bytecode compiler could open-code some common
> things like (looking-at "some constant").

lex.el's matcher (i.e. an Elisp loop interpetering the DFA
data-structure) is already pretty fast (i.e. competitive) in my
experience compared to the C based regex.c code.  So I think that
a C implementation of lex.el's matcher would already be so fast that
generating machine code for it would only make sense in extremely
rare circumstances.

> One "simple" way to improve regexp matching right now would be to remove
> the self-modifying code and change the implementation to use token
> threading, like we did for the bytecode interpreter.

Indeed, our regex.c code has a lot of room for improvement.  Given its
nasty worst case behavior, I've been reluctant to invest any more time
into it.

> I think removing this self-modifying stuff is also useful if we ever
> want to introduce first-class regexp objects.

It's also needed to make it re-entrant.


        Stefan