[Qemu-devel] [RFC] TCG unit testing

[Richard Henderson]

I've been thinking for a while about how to reliably test TCG backends, and
maybe how to do regression testing on them.  Having to begin the test from a
guest binary, especially considering the vast cross-compilation problem, is
pretty much a non-starter.

I've been thinking of a truly stripped down target for the purpose, with a
special-purpose machine loop and main to go with it.  I.e. avoid vl.c.

My current idea is that the test file consists of 3 sections: guest memory
layout, raw TBs, and expected results.  Perhaps best explained with some 
examples:

(1a) I've split up this test into two TBs to prevent some constant folding.
(1b) The guest machine should have enough registers to make it easy to perform
lots of tests at once.  Even better if we can avoid the complication of memory.

--------------------------------------------
# Test basic arithmetic, esp with constant arguments
CODE
0:      movi_i32        a0, 0x12345678
        movi_i32        a1, 0x87654321
        movi_tl         pc, 1
        exit_tb         0

1:      add_i32         a2, a1, a0
        sub_i32         a1, a1, a0
        movi_i32        t0, 1
        add_i32         a3, a0, t0
        sub_i32         a4, a0, t0
        movi_i32        t0, -1
        add_i32         a5, a0, t0
        sub_i32         a6, a0, t0
        movi_i32        t0, -0x5678
        add_i32         a7, a0, t0
        sub_i32         a8, a0, t0
        movi_i32        t0, 0x123456
        add_i32         a9, a0, t0
        sub_i32         a10, a0, t0
        movi_i32        t0, 0x12341234
        add_i32         a11, a0, t0
        sub_i32         a12, a0, t0
        call            done

RESULT
        a0      0x12345678
        a1      0x7530eca9
        a2      0x99999999
        a3      0x12345679
        a4      0x12345677
        a5      0x12345677
        a6      0x12345679
        a7      0x12340000
        a8      0x1234acf0
        a9      0x12468ace
        a10     0x12222222
        a11     0x246868ac
        a12     0x4444
-------------------------------------------

(2) The ram layout should be flexible enough to test specific scenarios

-------------------------------------------
# For 64-bit guest, 32-bit host, make sure we compare high-part of tlb
MEM
        ram     0x0_0000_0000, 4096
        ram     0x1_0000_0000, 4096

CODE
0:      movi_i64        b0, 0x12345678abcdef
        movi_i64        t1, 0
        movi_i64        t2, 0x1_0000_0000
        qemu_st32       b0, t1
        qemu_ld32       b1, t1
        qemu_ld32       b2, t2
        call            done

RESULT
        b0      0x12345678abcdef
        b1      0x12345678abcdef
        b2      0
-------------------------------------------

(3a) If we let "ioram" be defined via memory_region_init_io, with the "/l" and
"/b" qualifiers controlling the .endianness field of the region, then we can
test the memory access path all the way through.
(3b) The final test in the result section would *not* use the device interface
from memory.h, but rather examine the backing ram directly.

-------------------------------------------
MEM
        ioram/l 0, 4096                         # little-endian device
        ioram/b 4096, 4096                      # big-endian device
CODE
0:      movi_tl         t0, 0x12345678
        movi_tl         t1, 0
        qemu_st32       t0, t1
        qemu_ld32       a0, t1
        movi_tl         t1, 4096
        qemu_st32       t0, t1
        qemu_ld32       a1, t1
        call            done
RESULT
        a0      0x12345678
        a1      0x12345678
        mem/b   0, 0x78, 0x56, 0x34, 0x12       # little-endian bytes
        mem/b   4096, 0x12, 0x34, 0x56, 0x78    # big-endian bytes
-------------------------------------------

Thoughts?  Anything that we ought to be testing that I haven't thought of here,
and that this sort of structure couldn't support?

It may be some time before I can progress this enough to usefulness, but I
wanted to get this written down while it is fresh in my head.


r~