[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [PATCH RFC v2 04/16] vfio-user: connect vfio proxy to remote server
From: |
Stefan Hajnoczi |
Subject: |
Re: [PATCH RFC v2 04/16] vfio-user: connect vfio proxy to remote server |
Date: |
Mon, 13 Sep 2021 13:35:48 +0100 |
On Fri, Sep 10, 2021 at 05:25:13AM +0000, John Johnson wrote:
>
>
> > On Sep 8, 2021, at 11:29 PM, Stefan Hajnoczi <stefanha@redhat.com> wrote:
> >
> > On Thu, Sep 09, 2021 at 05:11:49AM +0000, John Johnson wrote:
> >>
> >>
> >> I did look at coroutines, but they seemed to work when the sender
> >> is triggering the coroutine on send, not when request packets are arriving
> >> asynchronously to the sends.
> >
> > This can be done with a receiver coroutine. Its job is to be the only
> > thing that reads vfio-user messages from the socket. A receiver
> > coroutine reads messages from the socket and wakes up the waiting
> > coroutine that yielded from vfio_user_send_recv() or
> > vfio_user_pci_process_req().
> >
> > (Although vfio_user_pci_process_req() could be called directly from the
> > receiver coroutine, it seems safer to have a separate coroutine that
> > processes requests so that the receiver isn't blocked in case
> > vfio_user_pci_process_req() yields while processing a request.)
> >
> > Going back to what you mentioned above, the receiver coroutine does
> > something like this:
> >
> > if it's a reply
> > reply = find_reply(...)
> > qemu_coroutine_enter(reply->co) // instead of signalling reply->cv
> > else
> > QSIMPLEQ_INSERT_TAIL(&pending_reqs, request, next);
> > if (pending_reqs_was_empty) {
> > qemu_coroutine_enter(process_request_co);
> > }
> >
> > The pending_reqs queue holds incoming requests that the
> > process_request_co coroutine processes.
> >
>
>
> How do coroutines work across threads? There can be multiple vCPU
> threads waiting for replies, and I think the receiver coroutine will be
> running in the main loop thread. Where would a vCPU block waiting for
> a reply? I think coroutine_yield() returns to its coroutine_enter() caller.
A vCPU thread holding the BQL can iterate the event loop if it has
reached a synchronous point that needs to wait for a reply before
returning. I think we have this situation when a MemoryRegion is
accessed on the proxy device.
For example, block/block-backend.c:blk_prw() kicks off a coroutine and
then runs the event loop until the coroutine finishes:
Coroutine *co = qemu_coroutine_create(co_entry, &rwco);
bdrv_coroutine_enter(blk_bs(blk), co);
BDRV_POLL_WHILE(blk_bs(blk), rwco.ret == NOT_DONE);
BDRV_POLL_WHILE() boils down to a loop like this:
while ((cond)) {
aio_poll(ctx, true);
}
I also want to check that I understand the scenarios in which the
vfio-user communication code is used:
1. vhost-user-server
The vfio-user communication code should run in a given AioContext (it
will be the main loop by default but maybe the user will be able to
configure a specific IOThread in the future).
2. vCPU thread vfio-user clients
The vfio-user communication code is called from the vCPU thread where
the proxy device executes. The MemoryRegion->read()/write() callbacks
are synchronous, so the thread needs to wait for a vfio-user reply
before it can return.
Is this what you had in mind?
Stefan
signature.asc
Description: PGP signature
- Re: [PATCH RFC v2 04/16] vfio-user: connect vfio proxy to remote server, Stefan Hajnoczi, 2021/09/07
- Re: [PATCH RFC v2 04/16] vfio-user: connect vfio proxy to remote server, John Johnson, 2021/09/09
- Re: [PATCH RFC v2 04/16] vfio-user: connect vfio proxy to remote server, Stefan Hajnoczi, 2021/09/09
- Re: [PATCH RFC v2 04/16] vfio-user: connect vfio proxy to remote server, John Johnson, 2021/09/10
- Re: [PATCH RFC v2 04/16] vfio-user: connect vfio proxy to remote server,
Stefan Hajnoczi <=
- Re: [PATCH RFC v2 04/16] vfio-user: connect vfio proxy to remote server, John Johnson, 2021/09/13
- Re: [PATCH RFC v2 04/16] vfio-user: connect vfio proxy to remote server, Stefan Hajnoczi, 2021/09/14
- Re: [PATCH RFC v2 04/16] vfio-user: connect vfio proxy to remote server, John Johnson, 2021/09/14
- Re: [PATCH RFC v2 04/16] vfio-user: connect vfio proxy to remote server, Stefan Hajnoczi, 2021/09/15
- Re: [PATCH RFC v2 04/16] vfio-user: connect vfio proxy to remote server, John Johnson, 2021/09/15
- Re: [PATCH RFC v2 04/16] vfio-user: connect vfio proxy to remote server, Stefan Hajnoczi, 2021/09/16