Re: [Duplicity-talk] Hanging just before completion

[Cody Ashe-McNalley]

Hi Ken,

The hanging seems to happen before upload would take place.  The sigtar files are huge, sometimes almost a terabyte.  But they will sit there for weeks with no network or process activity.  It only takes me a few hours to rename, gzip, and upload them.

It seems related to finishing the async process forking loop, but perhaps that does finish and it hangs immediately after.  The final difftar is uploaded.

Many thanks,

Cody

On Thu, Oct 15, 2020 at 6:36 AM Kenneth Loafman via Duplicity-talk <duplicity-talk@nongnu.org> wrote:

Hi Cody,

What are the sizes of the manifest and sigtar files?  They both get uploaded last and take a while.  Check for network activity.

...Thanks,

...Ken

 

On Wed, Oct 14, 2020 at 4:42 PM Cody Ashe-McNalley via Duplicity-talk <duplicity-talk@nongnu.org> wrote:

Hi All,

Duplicity is a great program and I appreciate this community and the development process.

Several duplicity jobs have hung after putting the final archive file up using the rclone backend.  The total archive size has been >=30TB.  The manifest and sigtar files are complete, but are left in the archive-dir with the .part suffix.  I've tried to look into this with gdb but the process forking makes that difficult.

Debug logging shows this as the point where it hangs:

DEBUG 1

. AsyncScheduler: active workers = 0

Centos Linux tethys.atmos.ucla.edu 3.10.0-1127.19.1.el7.x86_64
python 3.6.8 (virtual env)
duplicity-0.8.16
Options: --asynchronous-upload --no-compression --no-encryption --volsize 20000

I've been able to save the jobs by manually renaming and transferring the manifest and sigtar.  

Best regards,
Cody

_______________________________________________
Duplicity-talk mailing list
Duplicity-talk@nongnu.org
https://lists.nongnu.org/mailman/listinfo/duplicity-talk

_______________________________________________
Duplicity-talk mailing list
Duplicity-talk@nongnu.org
https://lists.nongnu.org/mailman/listinfo/duplicity-talk