Re: Questions about Disk I/O Record and Replay

[Pavel Dovgalyuk]

On 14.08.2021 10:25, Arnabjyoti Kalita wrote:
> Dear Pavel,
>
> Thank you again for the answers.
>
> I had another question -
>
> How is it possible to just use the request id while recording and use
> it during replay? Don't we have to record other parameters of the disk
> I/O request (like offset, bytes, iov buffer etc.) ?

Other parameters are deterministic in record/replay case, because they 
are generated by the guest code.

> Best Regards,
> Arnabjyoti Kalita
>
>
> On Fri, Aug 13, 2021 at 7:47 PM Pavel Dovgalyuk
> <pavel.dovgalyuk@ispras.ru> wrote:
> > On 13.08.2021 08:53, Arnabjyoti Kalita wrote:
> > > Hello all,
> > >
> > > I am trying to use the blkreplay driver to record and replay disk
> > > operations. My disk record and replay mechanism is a variant of the
> > > inbuilt record replay mechanism in QEMU.
> > >
> > > During record, I just store the completion ids of the disk operations
> > > as they happen. Something like this -
> > >
> > > void replay_block_event(QEMUBH *bh, uint64_t id)
> > > {
> > >       if (replay_mode == REPLAY_MODE_RECORD) {
> > >           if (start_recording) {
> > >               replay_put_qword(id, "disk");  // writes ids to a separate
> > > file dedicated for disk I/O record
> > >           }
> > >           qemu_bh_schedule(bh);
> > >       }
> > > }
> > >
> > > During replay, all I do is store the disk ID into an events list and
> > > ask the CPU to continue executing instructions.
> > >
> > > void replay_block_event(QEMUBH *bh, uint64_t id)
> > > {
> > >      if (replay_mode == REPLAY_MODE_RECORD) {
> > >          /* as shown above */
> > >      }
> > >      if (replay_mode == REPLAY_MODE_PLAY) {
> > >          BlockEvent *event = g_malloc0(sizeof(BlockEvent));
> > >          event->opaque = bh;
> > >          event->id = id;
> > >          QTAILQ_INSERT_TAIL(&blk_events_list, event, blk_events);
> > >          qemu_cpu_kick(first_cpu);                    <- Replayed guest
> > > gets stuck here
> > >      }
> > >      else {
> > >          qemu_bh_schedule(bh);
> > >      }
> > > }
> > >
> > > I know when the disk interrupt happens and so all the event ids that
> > > we store will be replayed right before that.
> > >
> > > My questions are as follows -
> > >
> > > 1. Is the above approach a good idea to achieve deterministic disk I/O replay?
> > >
> > > 2. Should I also replay disk event I/O as and when they arrive? How do
> > > I ensure that it actually gets completed before a checkpoint?
> >
> > I do not remember the details about this.
> >
> > > checkpoint in this case, is the disk interrupt.
> > >
> > > 3. How can I handle out-of-order disk I/O completions, in this
> > > scenario? How should I proceed?
> >
> > This is handled by saving events at the checkpoint.
> >
> > > I would love to get more details on these questions.
> > >
> > > Thank you very much for all your help so far.
> > >
> > > Best Regards,
> > > Arnabjyoti Kalita