Re: Large qcow2 performance

[Andrew Martin]

----- Original Message -----
> From: "Alberto Garcia" <address@hidden>
> To: "qemu-discuss" <address@hidden>
> Sent: Tuesday, September 24, 2019 10:15:56 AM
> Subject: Re: Large qcow2 performance

> On Mon, Sep 23, 2019 at 03:17:39PM -0500, Andrew Martin wrote:
> 
>> I am considering creating a VM with a large qcow2, around 6TB in
>> size. This Linux VM will be reading and writing a lot of small
>> files to this very large qcow2 using the VirtIO driver. Are there
>> any concerns or guidance about creating such a large qcow2, or
>> performance issues with small files on a 6TB qcow2?
> 
> Briefly:
> 
> there's two important parameters that can affect performance:
> cluster_size and l2_cache_size.
> 
> cluster_size is set when you create the image, l2_cache_size is a
> runtime option.
> 
> If the L2 cache size is not large enough, I/O performance will suffer.
> Check here for a summary (the defaults have changed since then, but
> the basic description of the problem is correct):
> 
>   
> https://blogs.igalia.com/berto/2015/12/17/improving-disk-io-performance-in-qemu-2-5-with-the-qcow2-l2-cache/
> 
> With small clusters you need a larger L2 cache, that is: QEMU needs
> more memory in order to get the same performance.
> 
> Images with large clusters use less memory, but there are two
> drawbacks:
> 
> - The cluster is the minimum unit of allocation, so even if you write
>  512 bytes QEMU needs to allocate one complete cluster and fill it
>  with data. This makes *allocation* much slower the larger the
>  cluster size.
> 
>  See the table here for an example, but focus only on the first and
>  last columns and ignore the two in the middle:
> 
>     https://lists.gnu.org/archive/html/qemu-block/2019-07/msg00496.html
> 
>  After a cluster is allocated an image with large clusters should
>  give you the same (or possibly better) performance.
> 
> - Because of what I just said, images with large clusters waste more
>  space on disk.
> 
> A hybrid approach that combines the benefits of large clusters
> regarding memory usage with the benefits of small clusters regarding
> allocation performance is currently under development, and I plan to
> send patches soon.
> 
> Berto

Hi Berto,

Thank you for the detailed explanation! It is also worth noting that I
am planning on hosting this qcow2 on a ZFS dataset (and ZFS does some
in-memory caching with the ARC as well); does that have an impact on
performance? 

Moreover, would setting the cluster_size to match the ZFS recordsize 
be advisable?
https://jrs-s.net/2018/03/13/zvol-vs-qcow2-with-kvm/

Thanks,

Andrew