duplicity-talk
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Duplicity-talk] Performance issues w/ S3


From: tink
Subject: Re: [Duplicity-talk] Performance issues w/ S3
Date: Wed, 24 Jul 2024 10:43:58 +1200



On Wed, 24 Jul 2024 at 01:10, edgar.soldin--- via Duplicity-talk <duplicity-talk@nongnu.org> wrote:
On 23.07.2024 04:13, tink via Duplicity-talk wrote:
> Encrypting a 10GB file (using compression) with GPG took  6m34s (again reading from rust, writing to SSD).
> gpg with no compression takes 37s

out of interest just played a bit on a local debian box with gpg options, which could be given as `--gpg-options` to duplicity
--compress-algo name
  https://www.gnupg.org/documentation/manuals/gnupg/GPG-Esoteric-Options.html#index-compress_002dalgo
--compress-level
n
  https://www.gnupg.org/documentation/manuals/gnupg/GPG-Configuration-Options.html#index-compress_002dlevel

tl;dr
gpg 2.2.40 only utilized one core
zip and zlib (gpg-default) are 6times slower than uncompressed
bzip2 is way way slower

tried zlib compression levels but couldn't determine a notable difference

in conclusion. so smaller volumes with duplicity 3.x `--concurrency` may be a way to saturate a broad upload bandwidth. but overall it still does not account for a 5MB/s throughout. even your 10GB/6m34s would be ruffly 25MB/s.

sunny regards ..ede
Thank you for the tests and the feedback ede ... 

I'll keep trying to find a working set-up w/ different python versions on Ubuntu 20.04 ... so far I haven't managed to get 3.0.0 to behave. 

reply via email to

[Prev in Thread] Current Thread [Next in Thread]