Re: A vm for native code in guile

[Nala Ginrut]

On Mon, 2013-01-14 at 09:43 +0100, Stefan Israelsson Tampe wrote:
> IMO, we don't have to write assembler again, since GNU Binutils does.
> The only necessary work is to map bytecode->asm, and add a AOT option
> with a script into 'guild' for calling Binutils.
> 
> 
> This is my approach: translate each opcode to a either a call to a c
> func or
> to a few assembller instructions. The translation to the c code is
> easy. But jumping and 
> moving data is best served by native instrucitons. Also we would like
> to effectivelly use machine registers
> ta gain perhaps an extra 2x in speed. This means quite a lot of work
> in assembler to move
> data around. To note we can gain some use of gcc by compiling for a
> restricted set of registers
> in order to be able to have a set of registers free for temporary
> storage. All this also means that we 
> can perhaps restrict the assembler to just include features for a very
> small set of instructions meaning that
> the assembler writing will not be as huge effort.
> 
Though write an simple x86-specific assembler won't be a difficult one,
I recommend reuse GNU code as much as possible.
And convert Scheme code to C code could be one of the way. It's nice to
give it a try. ;-)


> I'm not really sure how to use binutils as you suggest for the whole
> setup though. Perhaps you can explain in more detail.
> 
Binutils contains 'Assembler' & 'Linker' and other binary tools to
handle the native code generation.
I mention it here since we don't have to write assembler, the way I
suggest is to generate asm code only, then use binutls to make it an elf
one.

I think it's easier to implement a lisp-dialect compatible with
machine-description of GCC, like this:
https://android.googlesource.com/toolchain/gcc/+/57cfd7a03820d1241333e059feda0ab04829f575/gcc-4.6/gcc/config/i386/i386.md
And use this dialect to map registerVM-bytecode->asm. Then use 'as' to
assemble it. This way could avoid to re-implement assembler, and reuse
large of GCC code. 

> /Stefan
> 
> 
> On Mon, Jan 14, 2013 at 3:51 AM, Nala Ginrut <address@hidden>
> wrote:
>         On Wed, 2012-08-01 at 22:59 +0200, Stefan Israelsson Tampe
>         wrote:
>         > Hi,
>         >
>         > The byte-code -> native-code compiler is does serve my needs
>         pretty well
>         > now. It should really
>         > soon be possible to add code that will auto compile bytecode
>         versions to
>         > native versions. The compiler is not perfect and some
>         instructions is
>         > missing. But it can go from VM->NATIVE->VM and so on so
>         whenever there is
>         > missing instruction the compiler can bail out to vm code.
>         What's left is
>         > to be able to go from VM to Native returning multiple values
>         and in all
>         > call positions.
>         >
>         > To note
>         >
>         > * the code is for x86-64, linux.
>         >
>         > * Windows have another calling convention => the assembler
>         has to be recoded
>         >   => we need compilers for all interesting combinations of
>         operating
>         > systems and native targets
>         >
>         > * Using the C-stack is nice because the native push and pop
>         instructions
>         > can be used as well as
>         >   brk-ings makes for automatic stack growth? also calling
>         out c functions
>         > can be fast. On the
>         >   other hand stack traces is defunct with this code and I'm
>         uncertain how
>         > the prompting will
>         >   cope with this feature. It's probably better to use a
>         separate stack for
>         > the native code and model
>         >   it like the wip-rtl stack. On the other hand it has been
>         convenient to
>         > use it as a stack to save
>         >   variables before calling out to helper c-functions, but
>         these helper
>         > functions usually is on the slow
>         >   path and the savings can be done using helper registers
>         that is local to
>         > the vm a little bit slower
>         >   but doable. Not sure what path to take here.
>         >
>         > * Writing assembler is really tough. Because debugging is
>         really difficult.
>         >
>         
>         
>         IMO, we don't have to write assembler again, since GNU
>         Binutils does.
>         The only necessary work is to map bytecode->asm, and add a AOT
>         option
>         with a script into 'guild' for calling Binutils.
>         We may borrow some work from GCC. I don't know if it's easy,
>         but GCC
>         uses Lisp-like thing to handle machine-description. Though it
>         could be
>         interesting, it's a lot of work todo. Then it could support
>         many
>         platforms rather than x86.
>         
>         > * To ease things I compiled C code and examined the
>         assembler => fragile
>         > and difficult to port the
>         >   code. The final version needs to put more effort into
>         probing for
>         > constants used in the generated
>         >   assembler.
>         >
>         > * x86 code is pretty different because of the low number of
>         registers and
>         > invariant registers over c-call's
>         >
>         > * prompt and aborts are tricky instructions!
>         >
>         > Example:
>         > as an example reducing a list of 1000 elements with a
>         function that is
>         > basically + a 4x increase in performance when compiling to
>         native code
>         > could be seen. This are typical figures for what one can
>         expect to improve
>         > in speed. A smarter usage of registers and less poping and
>         pushing (RTL)
>         > could mean that we can increase the speedup from stable-2.0
>         even further.
>         >
>         >
>         > I will next week start working on the RTL branch porting the
>         current setup
>         > but use the rtl stack in stead of the native C stack.
>         >
>         > Regards
>         > /Stefan
>         
>         
>         
> 
>