Re: [Qemu-devel] virtqueue corruption in emulation mode?

[Stefan Hajnoczi]

On Wed, Sep 28, 2011 at 3:23 PM, Sinha, Ani <address@hidden> wrote:
>
> On Sep 28, 2011, at 1:51 AM, Stefan Hajnoczi wrote:
>
>> On Wed, Sep 28, 2011 at 3:01 AM, Sinha, Ani <address@hidden> wrote:
>>>
>>> On Sep 27, 2011, at 12:17 AM, Stefan Hajnoczi wrote:
>>>
>>>> On Mon, Sep 26, 2011 at 07:16:56PM -0500, Sinha, Ani wrote:
>>>>> I am using the virtqueue (virtqueue_pop, virtqueue_push etc) in the 
>>>>> emulated mode (non-kvm mode) from an IO thread (a separate thread 
>>>>> different from main QEMU thread). What I am observing is that the 
>>>>> virtqueue memory seems to get corrupt. Either qemu crashes while 
>>>>> performing virtqueue_push() (virtqueue_push() -> virtqueue_fill() 
>>>>> ->bring_used_idx()->lduw_phys()->qemu_get_ram_ptr()->"bad ram offset") or 
>>>>> crashes when the guest accesses a bad memory while using virtqueue. Now 
>>>>> this never ever happens when I run QEMU in KVM mode (/dev/kvm present) OR 
>>>>> when I use my functions from within the main qemu thread. I am unable to 
>>>>> figure out why this is happening. I have looked into my code over and 
>>>>> over again and I can't seem to explain this behavior. Can any of you guys 
>>>>> give me any inkling?
>>>>
>>>> QEMU is not thread-safe in general.  It uses a big lock to protect most
>>>> of its internal state.
>>>
>>>
>>> I see. So may be I should do something like qemu_set_fd_handler(fd, …) 
>>> where fd is a pipe and the handler does the virtqueue_push() etc?
>>> Now my question is, is it safe to do elem = virtqueue_pop(vq) from main 
>>> event loop, then so some work on the elem popped out in an worker thread 
>>> and then at some later point do a virtqueue_push(vq, elem) from that 
>>> handler (which is called by main_loop() ->main_loop_wait())?  In other 
>>> words, the vq reference is being used from the main event loop at two 
>>> different points from two different functions but not in a contiguous 
>>> fashion within the same function.
>>
>> Yes but do you need a helper thread?  Most of QEMU is based on
>> qemu_set_fd_handler() and callbacks, including for host network and
>> disk I/O.  If you follow the way QEMU does things it should be
>> easiest.
>
> I need a helper thread to do blocking IO. The device IOCTLS are inherently 
> blocking, unfortunately.

posix-aio-compat.c already implements a threadpool.  It is geared
towards using the QEMU block layer (BlockDriverState) but the pure
ioctl codepath can be used without a BlockDriverState by passing NULL.
 So this could save you some effort, check out paio_submit().

Stefan