Re: qemu-img convert vs writing another copy tool

[Markus Armbruster]

"Richard W.M. Jones" <address@hidden> writes:

> On Thu, Jan 23, 2020 at 07:53:57PM +0100, Max Reitz wrote:
>> On 23.01.20 19:35, Richard W.M. Jones wrote:
>> >  - NBD multi-conn.  In my tests this makes a really massive
>> >    performance difference in certain situations.  Again, virt-v2v has
>> >    a lot of information that we cannot pass to qemu: we know, for
>> >    example, exactly if the server supports the feature, how many
>> >    threads are available, in some situations even have information
>> >    about the network and backing disks that the data will travel over
>> >    / be stored on.
>> 
>> As far as I understand it, you use qemu-img convert with an NBD source
>> or target, too?
>
> Virt-v2v has many modes, but yes generally there will be either an NBD
> source & target, or an NBD source to a local file target.
>
>> I suppose it’s always easier to let a specialized and freshly written
>> tool handle such information.  But it sounds like if such information is
>> useful and makes that big of a difference, then it would be good to be
>> able to specify it to qemu’s NBD block driver, too.
>
> qemu-img convert has worked really well for us, and I'm actually _not_
> confident that I could do better with a specialized tool.  But there's
> definitely more info we could pass, such as the amount of parallelism
> we believe is available in the NBD server / processors / disks.
>
>> >  - Machine-parsable progress bars.  You can, sort of, parse the
>> >    progress bar from qemu-img convert, but it's not as easy as it
>> >    could be.  In particular it would be nice if the format was treated
>> >    as ABI, and if there was a way to have the tool write the progress
>> >    bar info to a precreated file descriptor.
>> 
>> It doesn’t seem impossible to add this feature to qemu-img, although I
>> wonder about the interface.  I suppose we could make it an alternative
>> progress output mode (with some command-line flag), and then the
>> information would be emitted to stdout (just like the existing progress
>> report).  You can of course redirect stdout to whatever fd you’d like,
>> so I don’t know whether qemu-img itself needs that specific capability.
>> 
>> OTOH, if you need this feature, why not just use qemu itself?  That is,
>> a mirror or a backup block job in an otherwise empty VM.
>
> I don't think we've really thought before about this approach.  Maybe
> the launching of a VM (even an empty / stopped one) could be a
> problem.  I guess this is what the new tool that was recently proposed
> upstream might help with?  (Was it called qemu-block-storage?  I can't
> find it right this minute)

Subject: [RFC PATCH 00/18] Add qemu-storage-daemon
To: address@hidden
Date: Thu, 17 Oct 2019 15:01:46 +0200
Message-Id: <address@hidden>

[...]