Re: [Duplicity-talk] Reducing local storage usage

[Kenneth Loafman]

Hi Rinck,

Looks like I had things backwards.  

Either way, lifecycle rules and duplicity do not play well together..  I think you should submit an issue to GitLab to get an option to ignore delete errors.  I'm not sure what the side effects will be, but that's the only solution I can think of since we are target agnostic and could not understand S3's issues.

...Thanks,

...Ken

On Tue, Sep 5, 2023 at 12:08 PM Rinck Sonnenberg <r.sonnenberg@netson.nl> wrote:

Hi Ken,

thanks for the quick reply. By syncing the lifecycle rules, do you mean setting the lifecycle rules to X days and using the remove-older-than with the same X days? In this particular case these 2 are indeed different, but wouldn't that create the same issue? My current local backup goes back to April this year, while my lifecycle rules delete files older than 30 days, so S3 does not contain any files older than 30 days, yet I still see the same error message?

Or did you mean something different? In that case, can you elaborate a bit?

Regards,

Rinck

On Tue, Sep 5, 2023 at 6:48 PM Kenneth Loafman via Duplicity-talk <duplicity-talk@nongnu.org> wrote:

Hi Rinck,

How about synching the lifecycle rules to your usage patterns.  That would be a simple solution.

Duplicity is aimed at "dumb" targets that only know how to (list, put, get, del), so what you're asking is a bit out-of-bounds for duplicity's intent.

If you disagree, please create an issue at GitLab.  Perhaps suggest an option "--ignore-failed-delete" to change the way we process errors.

...Thanks,

...Ken

On Tue, Sep 5, 2023 at 9:16 AM Rinck Sonnenberg via Duplicity-talk <duplicity-talk@nongnu.org> wrote:

Hi!

I've been using duplicity for many years and still love it! Keep up the good work! :-)

On some of my servers, the local duplicity cache is starting to take up some serious space and I'm trying to figure out what I can do to reduce this?

I have run the remove-older-than command to clean up some disk space, but am running into an issue there:

Deleting complete signature chain Wed Apr 12 00:20:02 2023
Attempt of _do_delete Nr. 1 failed. ClientError: An error occurred (AccessDenied) when calling the DeleteObject operation: Access Denied
Attempt of _do_delete Nr. 2 failed. ClientError: An error occurred (AccessDenied) when calling the DeleteObject operation: Access Denied
Attempt of _do_delete Nr. 3 failed. ClientError: An error occurred (AccessDenied) when calling the DeleteObject operation: Access Denied
Attempt of _do_delete Nr. 4 failed. ClientError: An error occurred (AccessDenied) when calling the DeleteObject operation: Access Denied
Attempt of _do_delete Nr. 5 failed. ClientError: An error occurred (AccessDenied) when calling the DeleteObject operation: Access Denied

.... etc

The access denied is a correct message from Amazon S3, since the backup user is not allowed to delete any files from the bucket. Instead, I use lifecycle rules on the bucket in combination with file prefixes to be able to delete older backups in S3 itself.

However, I would still like to be able to delete older local backups, without Duplicity attempting to delete the remote files as well. I've looked through the docs and manpage, but unless I missed something, it doesn't seem possible?

Is there a workaround I could use to accomplish this? Any help is very much appreciated!

Regards,

Rinck

_______________________________________________
Duplicity-talk mailing list
Duplicity-talk@nongnu.org
https://lists.nongnu.org/mailman/listinfo/duplicity-talk

_______________________________________________
Duplicity-talk mailing list
Duplicity-talk@nongnu.org
https://lists.nongnu.org/mailman/listinfo/duplicity-talk