Re: [Duplicity-talk] Old backups are not removed

[edgar . soldin]

hi Aitor,

comments inline below

On 17.10.2019 11:18, Aitor Soroa via Duplicity-talk wrote:
> Hi,
>
> I'm experiencing very similar problems as reported back in 2017 in the
> thread starting at
>
> https://lists.gnu.org/archive/html/duplicity-talk/2017-03/msg00014.html
>
> Namely, my old backups are not deleted, and therefore the remote disk
> gets full.
>
> At the beginning of each month I remove all the backups and create a
> full backup. Then, each day I create an incremental backup. The script
> I use is something like this:
>
> ---------------------------------------
> duplicity remove-older-than 3W --force file:////mnt/backup
> duplicity / --full-if-older-than 4W --include-filelist 
> /etc/duplicity/dirs.txt file:////mnt/backup
> ---------------------------------------

generally a bad idea conceptually. you should only run remove if your backups 
succeeded first.

> However, when the time comes it does not remove old backups:
>
> ---------------------------------------
>
> $ duplicity remove-older-than 3W --force file:////mnt/backup
> Local and Remote metadata are synchronized, no sync needed.
> Last full backup date: Thu Oct 17 00:05:59 2019
> There are backup set(s) at time(s):
> Wed Sep 18 08:54:46 2019
> Thu Sep 19 00:06:26 2019
> Fri Sep 20 00:07:16 2019
> Mon Sep 23 08:52:00 2019
> Tue Sep 24 00:07:06 2019
> Wed Sep 25 00:06:01 2019
> Thu Sep 26 00:09:06 2019
> Which can't be deleted because newer sets depend on them.
> No old backup sets found, nothing deleted.
> ---------------------------------------
>
> Collection-status gives me the following:
>
> ---------------------------------------
> $ duplicity collection-status file:///mnt/backup/
> Warning, found incomplete backup sets, probably left from aborted session
> Last full backup date: Wed Sep 18 08:54:46 2019
> Collection Status
> -----------------
> Connecting with backend: BackendWrapper
> Archive dir: /root/.cache/duplicity/870198592d381b892337283420a78807
>
> Found 0 secondary backup chains.
>
> Found primary backup chain with matching signature chain:
> -------------------------
> Chain start time: Wed Sep 18 08:54:46 2019
> Chain end time: Wed Oct 16 00:09:30 2019
> Number of contained backup sets: 27
> Total number of contained volumes: 105
>  Type of backup set:                            Time:      Num volumes:
>                 Full         Wed Sep 18 08:54:46 2019                79
>          Incremental         Thu Sep 19 00:06:26 2019                 1
>          Incremental         Fri Sep 20 00:07:16 2019                 1
>          Incremental         Mon Sep 23 08:52:00 2019                 1
>          Incremental         Tue Sep 24 00:07:06 2019                 1
>          Incremental         Wed Sep 25 00:06:01 2019                 1
>          Incremental         Thu Sep 26 00:09:06 2019                 1
>          Incremental         Fri Sep 27 00:07:06 2019                 1
>          Incremental         Sat Sep 28 00:09:06 2019                 1
>          Incremental         Sun Sep 29 00:06:00 2019                 1
>          Incremental         Mon Sep 30 00:09:22 2019                 1
>          Incremental         Tue Oct  1 00:05:35 2019                 1
>          Incremental         Wed Oct  2 00:07:00 2019                 1
>          Incremental         Thu Oct  3 00:09:03 2019                 1
>          Incremental         Fri Oct  4 00:09:46 2019                 1
>          Incremental         Sat Oct  5 00:10:22 2019                 1
>          Incremental         Sun Oct  6 00:07:15 2019                 1
>          Incremental         Mon Oct  7 00:10:44 2019                 1
>          Incremental         Tue Oct  8 00:09:50 2019                 1
>          Incremental         Wed Oct  9 00:09:43 2019                 1
>          Incremental         Thu Oct 10 00:06:54 2019                 1
>          Incremental         Fri Oct 11 00:07:10 2019                 1
>          Incremental         Sat Oct 12 00:10:31 2019                 1
>          Incremental         Sun Oct 13 00:07:16 2019                 1
>          Incremental         Mon Oct 14 00:08:52 2019                 1
>          Incremental         Tue Oct 15 00:05:52 2019                 1
>          Incremental         Wed Oct 16 00:09:30 2019                 1
> -------------------------
> Also found 0 backup sets not part of any chain,
> and 1 incomplete backup set.
> These may be deleted by running duplicity with the "cleanup" command.
> ---------------------------------------
>
> The "Warning, found incomplete backup sets, probably left from aborted
> session" above is that the last backup couldn't finish because the disk was
> full.
>
> Any help will be much appreciated!
>

looks good to me. as you keep a 4 week long chain duplicity cannot delete 
backups from 3 weeks ago and earlier as it still depends on them if you wanna 
restore eg. yesterdays backup.

in you current setup you will have to keep backing up successfully for at least 
7 weeks to see removal. duplicity will start a second chain and only after that 
was run for 3 weeks successfully the older chain can be removed. thats why it's 
called a chain ;)

technical background is simple. on restore duplicity first restores the first 
state of the file in the chain (usually from full) and the recursively applies 
the changes up until the date requested.

..ede/duply.net