Re: [PATCH 0/3] migration: add sztd compression

[Denis Plotnikov]

Hi, Juan

I'll read the series soon. Thanks for sending that to me!

Denis

On 24.01.2020 15:43, Juan Quintela wrote:
> Denis Plotnikov <address@hidden> wrote:
>> zstd date compression algorithm shows better performance on data compression.
>> It might be useful to employ the algorithm in VM migration to reduce CPU 
>> usage.
>> A user will be able to choose between those algorithms, therefor 
>> compress-type
>> migration parameter is added.
>>
>> Here are some results of performance comparison zstd vs gzip:
> Please, could you comment on the series:
>
> [PATCH v3 00/21] Multifd Migration Compression
>
> That series integrated zstd/zlib compression on top of multifd,
> advantages over "old" compression code are:
> - We don't have to copy data back and forth
> - The unit of compression is 512KB instead of 4kb
> - We "conserve" the compression state between packets (this is specially
>    interesting for zstd, that uses dictionaries)
>
>> host: i7-4790 8xCPU @ 3.60GHz, 16G RAM
>> migration to the same host
>> VM: 2xVCPU, 8G RAM total
>> 5G RAM used, memory populated with postgreqsl data
>> produced by pgbench performance benchmark
>>
>>
>> Threads: 1 compress – 1 decompress
>>
>> zstd provides slightly less compression ratio with almost the same
>> CPU usage but copes with RAM  compression roghly 2 times faster
>>
>> compression type              zlib       |      zstd
>> ---------------------------------------------------------
>> compression level          1       5     |   1       5
>> compression ratio          6.92    7.05  |   6.69    6.89
>> cpu idle, %                82      83    |   86      80
>> time, sec                  49      71    |   26      31
>> time diff to zlib, sec                      -25     -41
>>
>>
>> Threads: 8 compress – 2 decompress
>>
>> zstd provides the same migration time with less cpu consumption
>>
>> compression type         none  |        gzip(zlib)    |          zstd
>> ------------------------------------------------------------------------------
>> compression level        -     |  1      5       9    |   1       5       15
>> compression ratio        -     |  6.94   6.99    7.14 |   6.64    6.89    
>> 6.93
>> time, sec                154   |  22     23      27   |   23      23      25
>> cpu idle, %              99    |  45     30      12   |   70      52      23
>> cpu idle diff to zlib          |                      |  -25%    -22%    -11%
> I don't have handy results, but it looked for me like you:
> - zstd has a way better latency than zlib (i.e. the packet cames sooner)
> - And it compress much better
>
> On the migration test (best possible case for a compressor, as we are
> writting just one byte of each page, and we write the same value in all
> pages):
>
> - zlib: compress 512KB -> 2500 bytes
> - zstd: compess 512KB -> 52 bytes (yeap, I tested several times, it
>    looked too small)
>
> See that I posted another patch to "delete" the old compression code.
> Why?
> - I have been unable to modify migration-test to test it and work
>    reliablely (only way was to allow a really huge downtime)
> - Even with slow networking (1Gigabit) I got really mixed results,
>    because as it is so slow, the guest continue dirtying memory, and in
>    my tests it was never a winner
>
> Thanks, Juan.
>