Please make an issue of this at GitLab. duplicity should have stopped after
the failure to upload.
It may be due to the par2 wrapper. We'll know more once we have an issue to
work with.
...Thanks,
...Ken
On Wed, Aug 16, 2023 at 5:14 AM edgar.soldin--- via Duplicity-talk
<duplicity-talk@nongnu.org> wrote:
hey Jakob,
the relevant bit indeed is
> Giving up after 5 attempts. EndpointConnectionError: Could not
connect to the endpoint
URL:"https://wisemo-bk-hq.s3.eu-north-1.amazonaws.com/more/wisemo-bk-real-more_
duplicity-full.20230525T021330Z.vol272.difftar.gpg?uploads"
> Processed volume 273
> Writing
wisemo-bk-real-more_duplicity-full.20230525T021330Z.vol273.difftar.gpg
> Making directory
/bkup4/tmp/wisemo-bk-real-more/duplicity-720dor6j-tempdir/duplicity_temp.6
> Create Par2 recovery files
duplicity should not continue the backup if a retried upload
finally fails.
just checked the sources. the relevant code in the current
https://gitlab.com/duplicity/duplicity/-/blob/main/duplicity/backend.py#L40
1
but also in the 1.2.1 sources
https://gitlab.com/duplicity/duplicity/-/blob/rel.1.2.1/duplicity/backend.p
y?ref_type=tags#L419
calls log.FatalError(...)
https://gitlab.com/duplicity/duplicity/-/blob/main/duplicity/log.py?ref_typ
e=heads#L344
which in turn exits the program with an error code.
i am stumped why that didn't happen in your case. maybe because
of the par backend wrapper? looks like you found a bug. please
add a bug ticket
https://gitlab.com/duplicity/duplicity/-/issues
with these informations.
generally, as backups are crucial, i would advise to verify
regularily anyway, because even if an upload was supposedly
successful, it does not warrant any restorability. only
verification makes sure of that.
thanks.. ede/duply.net
On 16.08.2023 10:52, Jakob Bohm via Duplicity-talk wrote:
> Ahh, now I can confirm that the file doesn't exist.
>
> Digging into old logs for May 25, I see there were connection
errors for volumes 272 to 290, but somehow duplicity just kept
going past each one. I suspect some kind of transitional
network error during backup creation.
>
> But duplicity essentially ignored the errors and kept creating
and "uploading" volumes until the connection was restored and
later volumes got successfully uploaded.
>
> A few lines from that morning for volume 272 (this repeated
for 29 volumes):
>
> 58.3GB 06:04:56 [1.6MB/s]
[> ] 2% ETA 10d 6h 31min
> 58.3GB 06:04:59 [1.1MB/s]
[> ] 2% ETA 10d 6h 33min
> 58.3GB 06:05:02 [816.7KB/s]
[> ] 2% ETA 10d 6h 35min
> 58.3GB 06:05:05 [571.7KB/s]
[> ] 2% ETA 10d 6h 37min
> 58.3GB 06:05:08 [400.2KB/s]
[> ] 2% ETA 10d 6h 39min
> 58.3GB 06:05:11 [280.1KB/s]
[> ] 2% ETA 10d 6h 41min
> AsyncScheduler: scheduling task for asynchronous execution
> Processed volume 272
> Writing
wisemo-bk-real-more_duplicity-full.20230525T021330Z.vol272.difftar.gpg
> Making directory
/bkup4/tmp/wisemo-bk-real-more/duplicity-720dor6j-tempdir/duplicity_temp.1
> Create Par2 recovery files
> Reading results of 'par2 c -r10 -n1 -q
-q"/bkup4/tmp/wisemo-bk-real-more/duplicity-720dor6j-tempdir/duplicity_temp.1
/wisemo-bk-real-more_duplicity-full.20230525T021330Z.vol272.difftar.gpg"'
> 58.3GB 06:05:14 [196.1KB/s]
[> ] 2% ETA 10d 6h 43min
> 58.3GB 06:05:17 [137.3KB/s]
[> ] 2% ETA 10d 6h 45min
> 58.3GB 06:05:20 [96.1KB/s]
[> ] 2% ETA 10d 6h 47min
> 58.3GB 06:05:23 [67.3KB/s]
[> ] 2% ETA 10d 6h 49min
> 58.3GB 06:05:26 [47.1KB/s]
[> ] 2% ETA 10d 6h 51min
> A bkup3/MSSQL/Backup2015-12-03_SW6.bin
> 58.3GB 06:05:29 [33.0KB/s]
[> ] 2% ETA 10d 6h 53min
> 58.3GB 06:05:32 [23.1KB/s]
[> ] 2% ETA 10d 6h 55min
> 58.3GB 06:05:35 [16.1KB/s]
[> ] 2% ETA 10d 6h 57min
> A bkup3/MSSQL/Backup2015-12-03_SW7.bin
> 58.3GB 06:05:38 [11.3KB/s]
[> ] 2% ETA 10d 6h 59min
> 58.3GB 06:05:41 [7.9KB/s]
[> ] 2% ETA 10d 7h 1min
>
Deleting/bkup4/tmp/wisemo-bk-real-more/duplicity-720dor6j-tempdir/duplicity_temp.1/
wisemo-bk-real-more_duplicity-full.20230525T021330Z.vol272.difftar.gpg
>
Uploadingboto3+s3://wisemo-bk-hq/more/wisemo-bk-real-more_duplicity-full.20230525T02
1330Z.vol272.difftar.gpg to GLACIER_IR Storage
> 58.4GB 06:05:44 [1.3MB/s]
[> ] 2% ETA 10d 7h
> A bkup3/MSSQL/Backup2015-12-03_SW8.bin
> A bkup3/MSSQL/Backup2015-12-04_STAGING.bin
> A bkup3/MSSQL/Backup2015-12-04_SW1.bin
> 58.4GB 06:05:47 [4.4MB/s]
[> ] 2% ETA 10d 6h 53min
> A bkup3/MSSQL/Backup2015-12-04_SW2.bin
> 58.4GB 06:05:50 [3.3MB/s]
[> ] 2% ETA 10d 6h 55min
> 58.4GB 06:05:53 [2.3MB/s]
[> ] 2% ETA 10d 6h 57min
> 58.4GB 06:05:56 [1.6MB/s]
[> ] 2% ETA 10d 6h 59min
> 58.4GB 06:05:59 [1.1MB/s]
[> ] 2% ETA 10d 7h 1min
> 58.4GB 06:06:02 [815.7KB/s]
[> ] 2% ETA 10d 7h 3min
> 58.4GB 06:06:05 [570.9KB/s]
[> ] 2% ETA 10d 7h 5min
> 58.4GB 06:06:08 [399.6KB/s]
[> ] 2% ETA 10d 7h 7min
> 58.4GB 06:06:11 [279.6KB/s]
[> ] 2% ETA 10d 7h 9min
> 58.4GB 06:06:14 [195.8KB/s]
[> ] 2% ETA 10d 7h 11min
> 58.4GB 06:06:17 [137.0KB/s]
[> ] 2% ETA 10d 7h 13min
> 58.4GB 06:06:20 [95.9KB/s]
[> ] 2% ETA 10d 7h 15min
> 58.4GB 06:06:23 [67.1KB/s]
[> ] 2% ETA 10d 7h 17min
> A bkup3/MSSQL/Backup2015-12-04_SW6.bin
> 58.4GB 06:06:26 [47.0KB/s]
[> ] 2% ETA 10d 7h 19min
> A bkup3/MSSQL/Backup2015-12-04_SW7.bin
> 58.4GB 06:06:29 [32.9KB/s]
[> ] 2% ETA 10d 7h 21min
> AsyncScheduler: scheduling task for asynchronous execution
> 58.4GB 06:06:32 [23.0KB/s]
[> ] 2% ETA 10d 7h 23min
> 58.4GB 06:06:35 [16.1KB/s]
[> ] 2% ETA 10d 7h 25min
> 58.4GB 06:06:38 [11.3KB/s]
[> ] 2% ETA 10d 7h 27min
> 58.4GB 06:06:41 [7.9KB/s]
[> ] 2% ETA 10d 7h 29min
> 58.4GB 06:06:44 [5.5KB/s]
[> ] 2% ETA 10d 7h 31min
> 58.4GB 06:06:47 [3.9KB/s]
[> ] 2% ETA 10d 7h 33min
> 58.4GB 06:06:50 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:06:53 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:06:56 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:06:59 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:07:02 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:07:05 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:07:08 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:07:11 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:07:14 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:07:17 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:07:20 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:07:23 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:07:26 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:07:29 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:07:32 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:07:35 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:07:38 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:07:41 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:07:44 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:07:47 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:07:50 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:07:53 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:07:56 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:07:59 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:08:02 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:08:05 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:08:08 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:08:11 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:08:14 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:08:17 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:08:20 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:08:23 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:08:26 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:08:29 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:08:32 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:08:35 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:08:38 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:08:41 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:08:44 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:08:47 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:08:50 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:08:53 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:08:56 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:08:59 [0.0B/s]
[> ] 2% ETA Stalled!
> Attempt of put Nr. 1 failed. EndpointConnectionError: Could
not connect to the endpoint
URL:"https://wisemo-bk-hq.s3.eu-north-1.amazonaws.com/more/wisemo-bk-real-more_
duplicity-full.20230525T021330Z.vol272.difftar.gpg?uploadId=V0ycpw5Yj3a0Ow
y7H7BsoQv9OTAZ0I63uwIMZbPAfaY_yRTPV6Wi0zEHksAetOAs8w7b1BI336bTQHP5TBpWuJza
jmMsIRVXJZxd0Riu_lxMAg9dDaTQSE.cYrZazXWh&partNumber=1"
> 58.4GB 06:09:02 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:09:05 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:09:08 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:09:11 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:09:14 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:09:17 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:09:20 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:09:23 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:09:26 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:09:29 [0.0B/s]
[> ] 2% ETA Stalled!
> Writing
wisemo-bk-real-more_duplicity-full.20230525T021330Z.vol272.difftar.gpg
> Making directory
/bkup4/tmp/wisemo-bk-real-more/duplicity-720dor6j-tempdir/duplicity_temp.2
> Create Par2 recovery files
> Reading results of 'par2 c -r10 -n1 -q
-q"/bkup4/tmp/wisemo-bk-real-more/duplicity-720dor6j-tempdir/duplicity_temp.2
/wisemo-bk-real-more_duplicity-full.20230525T021330Z.vol272.difftar.gpg"'
> 58.4GB 06:09:32 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:09:35 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:09:38 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:09:41 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:09:44 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:09:47 [0.0B/s]
[> ] 2% ETA Stalled!
>
Deleting/bkup4/tmp/wisemo-bk-real-more/duplicity-720dor6j-tempdir/duplicity_temp.2/
wisemo-bk-real-more_duplicity-full.20230525T021330Z.vol272.difftar.gpg
>
Uploadingboto3+s3://wisemo-bk-hq/more/wisemo-bk-real-more_duplicity-full.20230525T02
1330Z.vol272.difftar.gpg to GLACIER_IR Storage
> 58.4GB 06:09:50 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:09:53 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:09:56 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:09:59 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:10:02 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:10:05 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:10:08 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:10:11 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:10:14 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:10:17 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:10:20 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:10:23 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:10:26 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:10:29 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:10:32 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:10:35 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:10:38 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:10:41 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:10:44 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:10:47 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:10:50 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:10:53 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:10:56 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:10:59 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:11:02 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:11:05 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:11:08 [0.0B/s]
[> ] 2% ETA Stalled!
> Attempt of put Nr. 2 failed. EndpointConnectionError: Could
not connect to the endpoint
URL:"https://wisemo-bk-hq.s3.eu-north-1.amazonaws.com/more/wisemo-bk-real-more_
duplicity-full.20230525T021330Z.vol272.difftar.gpg?uploads"
> 58.4GB 06:11:11 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:11:14 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:11:17 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:11:20 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:11:23 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:11:26 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:11:29 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:11:32 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:11:35 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:11:38 [0.0B/s]
[> ] 2% ETA Stalled!
> Writing
wisemo-bk-real-more_duplicity-full.20230525T021330Z.vol272.difftar.gpg
> Making directory
/bkup4/tmp/wisemo-bk-real-more/duplicity-720dor6j-tempdir/duplicity_temp.3
> Create Par2 recovery files
> Reading results of 'par2 c -r10 -n1 -q
-q"/bkup4/tmp/wisemo-bk-real-more/duplicity-720dor6j-tempdir/duplicity_temp.3
/wisemo-bk-real-more_duplicity-full.20230525T021330Z.vol272.difftar.gpg"'
> 58.4GB 06:11:41 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:11:44 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:11:47 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:11:50 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:11:53 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:11:56 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:11:59 [0.0B/s]
[> ] 2% ETA Stalled!
>
Deleting/bkup4/tmp/wisemo-bk-real-more/duplicity-720dor6j-tempdir/duplicity_temp.3/
wisemo-bk-real-more_duplicity-full.20230525T021330Z.vol272.difftar.gpg
>
Uploadingboto3+s3://wisemo-bk-hq/more/wisemo-bk-real-more_duplicity-full.20230525T02
1330Z.vol272.difftar.gpg to GLACIER_IR Storage
> 58.4GB 06:12:02 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:12:05 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:12:08 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:12:11 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:12:15 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:12:18 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:12:21 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:12:24 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:12:27 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:12:30 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:12:33 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:12:36 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:12:39 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:12:42 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:12:45 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:12:48 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:12:51 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:12:54 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:12:57 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:13:00 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:13:03 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:13:06 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:13:09 [0.0B/s]
[> ] 2% ETA Stalled!
> Attempt of put Nr. 3 failed. EndpointConnectionError: Could
not connect to the endpoint
URL:"https://wisemo-bk-hq.s3.eu-north-1.amazonaws.com/more/wisemo-bk-real-more_
duplicity-full.20230525T021330Z.vol272.difftar.gpg?uploads"
> 58.4GB 06:13:12 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:13:15 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:13:18 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:13:21 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:13:24 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:13:27 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:13:30 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:13:33 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:13:36 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:13:39 [0.0B/s]
[> ] 2% ETA Stalled!
> Writing
wisemo-bk-real-more_duplicity-full.20230525T021330Z.vol272.difftar.gpg
> Making directory
/bkup4/tmp/wisemo-bk-real-more/duplicity-720dor6j-tempdir/duplicity_temp.4
> Create Par2 recovery files
> Reading results of 'par2 c -r10 -n1 -q
-q"/bkup4/tmp/wisemo-bk-real-more/duplicity-720dor6j-tempdir/duplicity_temp.4
/wisemo-bk-real-more_duplicity-full.20230525T021330Z.vol272.difftar.gpg"'
> 58.4GB 06:13:42 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:13:45 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:13:48 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:13:51 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:13:54 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:13:57 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:14:00 [0.0B/s]
[> ] 2% ETA Stalled!
>
Deleting/bkup4/tmp/wisemo-bk-real-more/duplicity-720dor6j-tempdir/duplicity_temp.4/
wisemo-bk-real-more_duplicity-full.20230525T021330Z.vol272.difftar.gpg
>
Uploadingboto3+s3://wisemo-bk-hq/more/wisemo-bk-real-more_duplicity-full.20230525T02
1330Z.vol272.difftar.gpg to GLACIER_IR Storage
> 58.4GB 06:14:03 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:14:06 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:14:09 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:14:12 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:14:15 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:14:18 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:14:21 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:14:24 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:14:27 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:14:30 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:14:33 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:14:36 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:14:39 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:14:42 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:14:45 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:14:48 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:14:51 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:14:54 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:14:57 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:15:00 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:15:03 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:15:06 [0.0B/s]
[> ] 2% ETA Stalled!
> Attempt of put Nr. 4 failed. EndpointConnectionError: Could
not connect to the endpoint
URL:"https://wisemo-bk-hq.s3.eu-north-1.amazonaws.com/more/wisemo-bk-real-more_
duplicity-full.20230525T021330Z.vol272.difftar.gpg?uploads"
> 58.4GB 06:15:09 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:15:12 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:15:15 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:15:18 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:15:21 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:15:24 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:15:27 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:15:30 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:15:33 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:15:36 [0.0B/s]
[> ] 2% ETA Stalled!
> Writing
wisemo-bk-real-more_duplicity-full.20230525T021330Z.vol272.difftar.gpg
> Making directory
/bkup4/tmp/wisemo-bk-real-more/duplicity-720dor6j-tempdir/duplicity_temp.5
> Create Par2 recovery files
> Reading results of 'par2 c -r10 -n1 -q
-q"/bkup4/tmp/wisemo-bk-real-more/duplicity-720dor6j-tempdir/duplicity_temp.5
/wisemo-bk-real-more_duplicity-full.20230525T021330Z.vol272.difftar.gpg"'
> 58.4GB 06:15:39 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:15:42 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:15:45 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:15:48 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:15:51 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:15:54 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:15:57 [0.0B/s]
[> ] 2% ETA Stalled!
>
Deleting/bkup4/tmp/wisemo-bk-real-more/duplicity-720dor6j-tempdir/duplicity_temp.5/
wisemo-bk-real-more_duplicity-full.20230525T021330Z.vol272.difftar.gpg
>
Uploadingboto3+s3://wisemo-bk-hq/more/wisemo-bk-real-more_duplicity-full.20230525T02
1330Z.vol272.difftar.gpg to GLACIER_IR Storage
> 58.4GB 06:16:00 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:16:03 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:16:06 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:16:09 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:16:12 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:16:15 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:16:18 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:16:21 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:16:24 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:16:27 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:16:30 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:16:33 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:16:36 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:16:39 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:16:42 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:16:45 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:16:48 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:16:51 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:16:54 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:16:57 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:17:00 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:17:03 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:17:06 [0.0B/s]
[> ] 2% ETA Stalled!
> Giving up after 5 attempts. EndpointConnectionError: Could not
connect to the endpoint
URL:"https://wisemo-bk-hq.s3.eu-north-1.amazonaws.com/more/wisemo-bk-real-more_
duplicity-full.20230525T021330Z.vol272.difftar.gpg?uploads"
> Processed volume 273
> Writing
wisemo-bk-real-more_duplicity-full.20230525T021330Z.vol273.difftar.gpg
> Making directory
/bkup4/tmp/wisemo-bk-real-more/duplicity-720dor6j-tempdir/duplicity_temp.6
> Create Par2 recovery files
> Reading results of 'par2 c -r10 -n1 -q
-q"/bkup4/tmp/wisemo-bk-real-more/duplicity-720dor6j-tempdir/duplicity_temp.6
/wisemo-bk-real-more_duplicity-full.20230525T021330Z.vol273.difftar.gpg"'
> 58.4GB 06:17:09 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:17:12 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:17:15 [0.0B/s]
[> ] 2% ETA Stalled!
> 58.4GB 06:17:18 [0.0B/s]
[> ] 2% ETA Stalled!
>
>
>
> On 2023-08-15 23:57, Scott Hannahs wrote:
>> Before you go ragging on duplicity too much, it appears that
this issue is a file is missing from your storage. This is more
a storage issue than a duplicity issue. It is not clear that
duplicity is the source of your missing volume. Network
transfer issues may also be at play. Using the duplicity
“verify” functionality might warn you of a storage issue.
>>
>> The file isprobably
wisemo-bk-real-more_duplicity-full.20230525T021330Z.vol272.difftar
.gpg as my best guess at your backup strategy.
>>
>> I admit the “KeyError” report is a little hard to fathom as
to what is the issue.
>>
>> I am not sure how big your backup is but 11K volumes seems a
bit large, you might want to increase the volume size to reduce
the number of volumes.
>>
>> Duplicity is designed to work with insecure and unreliable
storage, but to fully implement that you need to have multiple
copies in multiple places.
>>
>> -sth
>>
>>> On Aug 15, 2023, at 5:15 PM, Jakob Bohm via Duplicity-talk
<duplicity-talk@nongnu.org> wrote:
>>>
>>> Dear list,
>>>
>>> This path contains a chain of full and incremental backups
going back
>>> months. Any competent backup program is expected to not
corrupt its
>>> storage and then proceed without errors to add further
incrementals
>>> in a way that it cannot restore .
>>>
>>> As to how to check the directory listing on AWS S3, I don't
know what
>>> file name to look for, especially given the limitation that
S3 can only
>>> search on a name prefix, not arbitrary wildcards.
>>>
>>> The partition in question contains terabytes of data, that
should not
>>> be lost under any circumstance, and this is the fire drill
test of the
>>> restore procedure. If duplicity cannot handle this basic
task, I will
>>> need to look for other solutions for continuous system
backup.
>>>
>>>
>>>
>>> On 2023-08-11 18:54, Kenneth Loafman via Duplicity-talk
wrote:
>>>> That specific error says that volume 272 difftar of the
backupset is missing. Not a good error message by any means.
>>>>
>>>> You can check on S3 to see if somehow it did not get
downloaded, but my guess is that it is indeed missing.
>>>>
>>>> Did you have any restarts during the backup? Did you
verify the backup?
>>>>
>>>> ...Ken
>>>>
>>>>
>>>> On Fri, Aug 11, 2023 at 1:17 AM Jakob Bohm via
Duplicity-talk <duplicity-talk@nongnu.org
<mailto:duplicity-talk@nongnu.org>> wrote:
>>>>
>>>> Dear all,
>>>>
>>>> I am testing the viability of my backups by restoring
to a blank
>>>> Linux
>>>> computer loaded up with the decryption keys and the
same patched
>>>> duplicity 1.2.1 that I used for the backup (just a
patch for the
>>>> doubled
>>>> --encrypt option and one for lowering the threshold for
displaying
>>>> actual progress numbers instead of "Stalled").
>>>>
>>>> However when restoring a large partition, instead of
restoring,
>>>> it fails
>>>> with the error messages below, the message doesn't say
what the
>>>> error is
>>>> or what it was trying to do, just an error number and
traceback
>>>> of the
>>>> stack with no specific context:
>>>>
>>>> Added incremental Backupset (start_time: Thu May 25
02:13:30 2023 /
>>>> end_time: Thu Jun 22 11:50:21 2023)
>>>> Added incremental Backupset (start_time: Thu Jun 22
11:50:21 2023 /
>>>> end_time: Mon Jul 3 09:17:24 2023)
>>>> Added incremental Backupset (start_time: Mon Jul 3
09:17:24 2023 /
>>>> end_time: Mon Jul 17 18:50:05 2023)
>>>> Added incremental Backupset (start_time: Mon Jul 17
18:50:05 2023 /
>>>> end_time: Wed Jul 19 22:32:43 2023)
>>>> Added incremental Backupset (start_time: Wed Jul 19
22:32:43 2023 /
>>>> end_time: Mon Jul 24 03:50:01 2023)
>>>> Added incremental Backupset (start_time: Mon Jul 24
03:50:01 2023 /
>>>> end_time: Mon Aug 7 21:33:50 2023)
>>>> Local and Remote metadata are synchronized, no sync
needed.
>>>> Last full backup date: Thu May 25 02:13:30 2023
>>>> Processing local manifest
>>>>
b'/bkup4/archive/wisemo-bk-real-more/wisemo-bk-real-more/wisemo-bk-real-mor
e_duplicity-full.20230525T021330Z.manifest'
>>>>
>>>> (82148726)
>>>> Found 11293 volumes in manifest
>>>> Traceback (innermost last):
>>>> File "/usr/local/bin/duplicity", line 86, in
<module>
>>>> with_tempdir(main)
>>>> File "/usr/local/bin/duplicity", line 70, in
with_tempdir
>>>> fn()
>>>> File
>>>>
"/usr/local/lib/python3.11/dist-packages/duplicity/dup_main.py",
>>>> line 1575, in main
>>>> do_backup(action)
>>>> File
>>>>
"/usr/local/lib/python3.11/dist-packages/duplicity/dup_main.py",
>>>> line 1658, in do_backup
>>>> restore(col_stats)
>>>> File
>>>>
"/usr/local/lib/python3.11/dist-packages/duplicity/dup_main.py",
>>>> line 749, in restore
>>>> restore_get_patched_rop_iter(col_stats)):
>>>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>>>> File
>>>>
"/usr/local/lib/python3.11/dist-packages/duplicity/dup_main.py",
>>>> line 801, in restore_get_patched_rop_iter
>>>> file_names.append(backup_set.volume_name_dict[vol_num])
>>>> ~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^
>>>> KeyError: 272
>>>>
>>>>
>
>
> Enjoy
>
> Jakob
_______________________________________________
Duplicity-talk mailing list
Duplicity-talk@nongnu.org
https://lists.nongnu.org/mailman/listinfo/duplicity-talk