Re: [Duplicity-talk] sigtar file: AccessDenied error using S3 Glacier after upgrade to 0.8.23

[edgar . soldin]

hey Bernhard,

On 15.07.2022 07:06, Bernhard Patsch via Duplicity-talk wrote:
> Hello,
>
> I use duplicity with an S3 Glacier bucket from Scaleway. After upgrading to 
> 0.8.23 and enabling boto3, the backup fails with the following error:
>
> % /usr/bin/duplicity -v3 --full-if-older-than 1M --s3-use-new-style --s3-use-glacier 
> --s3-endpoint-url https://s3.nl-ams.scw.cloud <https://s3.nl-ams.scw.cloud> 
> --s3-region-name nl-ams --asynchronous-upload --s3-multipart-chunk-size 10 
> --metadata-sync-mode partial --exclude-device-files --include=/etc --include=/data/Pictures 
> '--exclude=**' / boto3+s3://<bucket-name>/duplicity
>
> Synchronizing remote metadata to local cache...
>  >>> Copying duplicity-full-signatures.20220611T132657Z.sigtar.gpg to local 
> cache.
>  >>> Attempt of get Nr. 1 failed. ClientError: An error occurred (AccessDenied) when 
> calling the >>> GetObject operation: Access Denied.
>
> I understand that the current boto3 implementation does not support restore 
> from Glacier storage.

changing the target url, changes '--archive-dir' path, so a new one needs to be 
downloaded. that's halfway to a restore on a different machine.
 
> According to the man page, the manifest file is always stored in standard class.

yes, because only them are needed for a successful incremental if archive-dir 
is intact.

> However, the sigtar file is stored in Glacier class when the option 
> s3_use_clacier is set.
> The error is gone when the sigtar file is manually restored to standard class.

that's indeed needed. check the comment in the backend source
https://gitlab.com/duplicity/duplicity/-/blob/main/duplicity/backends/s3_boto3_backend.py#L31-43
 
> I assume that the storage class should be set explicitely for the sigtar file, 
> similar to the handling for the manifest in s3_boto3_backend.py:123++?
> If this assumption is correct, I can raise a bug ticket for that problem.

i'd say it is a bit but not fully. a described above the sig is not strictly 
needed and might become quite big over time as it is not split currently.

sunny regards ..ede/duply.net