[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [Duplicity-talk] Verification of filelists, multiprocessing
From: |
edgar . soldin |
Subject: |
Re: [Duplicity-talk] Verification of filelists, multiprocessing |
Date: |
Thu, 28 Jan 2016 11:45:42 +0100 |
User-agent: |
Mozilla/5.0 (Windows NT 5.1; rv:38.0) Gecko/20100101 Thunderbird/38.5.1 |
On 28.01.2016 01:35, Cláudio Gil wrote:
> Em 27/01/2016 22:27, "Andreas Doll" <address@hidden> escreveu:
>>
>> On 2016-01-27 at 21:27, Cláudio Gil wrote:
>>> Maybe paralelization could be introduced in a few select points of
> duplicity.
>>> But the part that is mostly CPU-bound is the encryption and encrypting
> a file
>>> will use 1 core.
>>
>> Yes, my naive idea was that it may be possible to de/encrypt files in
>> parallel. However, I'm not familiar with duplicitys codebase, and I'm not
> sure
>> if this is feasible.
>
> Multiple volumes can be encrypted or decrypted in parallel but, regardless
> of being easy or hard to change duplicity to do it, I'm not even sure it's
> worth it. There is a lot of IO involved, for example, and volumes still
> need to be processed in a sequence, by duplicity.
>
biggest bottleneck (in terms of speed) currently is probably the sequential
upload to the backend. if someone would completely asynchronize that from
backup creation, so duplicity could generate away and files would be uploaded
independently in another thread, maybe even in parallel if there are enough
volumes finished, that would make a dent i guess.
ede/duply.net