Re: [RFC v2] migration: Add migrate-set-bitmap-node-mapping

[Vladimir Sementsov-Ogievskiy]

[add Nikolay]

18.05.2020 19:26, Peter Krempa wrote:
> On Wed, May 13, 2020 at 16:56:10 +0200, Max Reitz wrote:
> > This command allows mapping block node names to aliases for the purpose
> > of block dirty bitmap migration.
> >
> > This way, management tools can use different node names on the source
> > and destination and pass the mapping of how bitmaps are to be
> > transferred to qemu (on the source, the destination, or even both with
> > arbitrary aliases in the migration stream).
> >
> > Suggested-by: Vladimir Sementsov-Ogievskiy <address@hidden>
> > Signed-off-by: Max Reitz <address@hidden>
> > ---
>
> [...]
>
> > ---
> >   qapi/migration.json            | 36 ++++++++++++++++++++
> >   migration/block-dirty-bitmap.c | 60 ++++++++++++++++++++++++++++++++--
> >   2 files changed, 94 insertions(+), 2 deletions(-)
>
> I just started to write some quick and dirty hacks to test use of this
> infrastructure in libvirt. I have 2 quick observations for now:
>
> > diff --git a/qapi/migration.json b/qapi/migration.json
> > index d5000558c6..97037ea635 100644
> > --- a/qapi/migration.json
> > +++ b/qapi/migration.json
> > @@ -1621,3 +1621,39 @@
> >   ##
> >   { 'event': 'UNPLUG_PRIMARY',
> >     'data': { 'device-id': 'str' } }
> > +
> > +##
> > +# @MigrationBlockNodeMapping:
> > +#
> > +# Maps a block node name to an alias for migration.
> > +#
> > +# @node-name: A block node name.
> > +#
> > +# @alias: An alias name for migration (for example the node name on
> > +#         the opposite site).
> > +#
> > +# Since: 5.1
> > +##
> > +{ 'struct': 'MigrationBlockNodeMapping',
> > +  'data': {
> > +      'node-name': 'str',
> > +      'alias': 'str'
> > +  } }
>
> We'd probably like a
> 'nodename:bitmapname' -> 'alias' mapping so that we can select which

If we need bitmap mapping, than it should be nodename:bitmapname -> 
nodealias:bitmapalias mapping, for backward-compatibility with current qemu.

> bitmaps to migrate and where to migrate them to. The specific use case
> is following:
>
> Libvirt supports migration without shared storage (NFS/etc) where we
> copy the disk images prior to the memory migration using qemu's NBD
> server and the blockdev-mirror job. By default and the most simple way
> which doesn't require users fudging with the disk images and copying
> backing images themselves is that we flatten all the backing chain
> during the copy ("sync":"full"). In this mode we'll need to do some
> merging of the bitmaps prior to finalizing the copy.
>
> It's not a problem to do it ourselves, but in the end we'll need to copy
> only certain bitmaps which will be created temporarily on the source to
> the destination where they'll be persisted.
>
> For now (until I implement the use of the dirty-bitmap-populate blockjob
> which I'm also doing in parallel with this kind of) when creating a
> snapshot we create a new active bitmap in the overlay for every active
> bitmap in the snapshotted image.
>
> When flattening we'll then need to merge the appropriate ones. As said
> it's not a problem to prepare all the bitmaps before but we then don't
> need to migrate all of them.
>
> By the way, that brings me to another question:
>
> Is there any difference of handling of persistent and non-persistent
> bitmaps? Specifically I'm curious what's the correct approach to do the
> shared storage migration case when the source and destination image is
> the same one. Should we also instruct to migrate the active ones? or are
> they migrated by writing them to the image and reloading them?

About migration of persistent bitmaps with shared storage: both variants are 
possible:

1. if you do nothing (i.e. not enable bitmaps migration capability), persistent 
bitmaps are stored on inactivation of source Qemu and loaded on activation of 
target Qemu

2. if you enable bitmap migration capability, then bitmaps are _NOT_ stored, 
they migrate through migration channel

The difference is in downtime: if we have a lot of bitmap data (big disks, 
small bitmap granularity, a lot of bitmaps) and/or slow shared storage, then 
with [1] downtime will increase, as we'll have to store all bitmaps to the disk 
and load them during migration downtime. With [2] bitmaps migrate in postcopy 
time, when target is already running, so downtime is not increased.

So, in general [2] is better, and I think we should always use it, not making 
extra difference between shared and non-shared migration.

==

And in this way, no difference between persistent and non-persistent bitmaps 
migration, let's always migrate them by postcopy [and we go this way (I hope ;) 
in Virtuozzo]

> > +##
> > +# @migrate-set-bitmap-node-mapping:
>
> qemu-5.0 deprecated a bunch of migrate-set- specific commands in favor
> of migrate-set-parameters. Is it worth/necessary to add a new command
> here?

Hmm probably, we should include mapping into MigrateSetParameters structure?

> > +#
> > +# Maps block node names to arbitrary aliases for the purpose of dirty
> > +# bitmap migration.  Such aliases may for example be the corresponding
> > +# node names on the opposite site.
> > +#
> > +# By default, every node name is mapped to itself.
> > +#
> > +# @mapping: The mapping; must be one-to-one, but not necessarily
> > +#           complete.  Any mapping not given will be reset to the
> > +#           default (i.e. the identity mapping).
> > +#
> > +# Since: 5.1
> > +##


--
Best regards,
Vladimir