[Duplicity-talk] how to generate a full backup from incremental backups and how to list files of an incremental backup

[Jelle de Jong]

Hello everybody,

I have been using duplicity for a several months now on different
networks with different set-ups. I am very pleased with the
flexibility and reliability of duplicity. So thank you for the tool
and I hope the development continues to create stable secure software.

I do have a few question:

1) How do I generate a list of the changed/backup files in an
incremental backup? (I want to know what the incremental backup did)

2) I have a few set-ups with very minimal upload bandwidth, and I want
to stay using incremental backups as standard setting without periodic
full backups. So I want duplicity to clean-up the backup archive every
month or so, to generate a new full backup from all the incremental
backups it done in the past. How can I do this? So I don't want to do
an "--full-if-older-than" but some sort of
"--merge-incremental-if-older-then" and if possible say to create a
full backup from increment backup older then 15 days?

3) See the attachment, for the dupplicity run i made for one of my
networks, is there anything "not so smart" about doing a backup this way?

Thanks in advance,

Kind regards,

Jelle de Jong

#!/bin/bash

# Company:  PowerCraft Technology
# Author:   Copyright Jelle de Jong <address@hidden>
# Note:     Please send me an email if you enhanced the document
# Date:     2010-01-31
# License:  CC-BY-SA

export PASSPHRASE=""

echo "phase 0: home: "
duplicity remove-older-than 3M --force \
scp://address@hidden/backup/home

echo "phase 1: vmail: "
duplicity remove-older-than 3M --force \
scp://address@hidden/backup/vmail

echo "phase 2: root: "
duplicity remove-older-than 3M --force \
scp://address@hidden/backup/root

echo "phase 3: home: "
duplicity --encrypt-key "581B2D59" --sign-key "B22E82DB" \
--full-if-older-than 30D \
--exclude /home/management/backup \
--exclude /home/vmail \
--exclude /home/system02-backup \
--scp-command "scp -C -c blowfish -l 320" \
/home scp://address@hidden/backup/home

echo "phase 4: vmail: "
duplicity --encrypt-key "581B2D59" --sign-key "B22E82DB" \
--full-if-older-than 38D \
--exclude /home/vmail/system/backup/somebody.com \
--scp-command "scp -C -c blowfish -l 320" \
/home/vmail scp://address@hidden/backup/vmail

echo "phase 5: root: "
duplicity --encrypt-key "581B2D59" --sign-key "B22E82DB" \
--full-if-older-than 46D \
--exclude /proc --exclude /mnt --exclude /tmp --exclude /sys \
--exclude /dev --exclude /home \
--scp-command "scp -C -c blowfish -l 320" \
/ scp://address@hidden/backup/root

echo "phase 6: diskfree: "
ssh address@hidden df -h /home

exit

phase 0: home:
Local and Remote metadata are synchronized, no sync needed.
Last full backup date: Sat Jan 30 23:30:04 2010
No old backup sets found, nothing deleted.

phase 1: vmail:
Local and Remote metadata are synchronized, no sync needed.
Warning, found the following remote orphaned signature files:
duplicity-new-signatures.20100203T223035Z.to.20100204T223041Z.sigtar.gpg
duplicity-new-signatures.20100204T223041Z.to.20100205T223035Z.sigtar.gpg
duplicity-new-signatures.20100205T223035Z.to.20100206T223034Z.sigtar.gpg
duplicity-new-signatures.20100206T223034Z.to.20100207T223036Z.sigtar.gpg
duplicity-new-signatures.20100207T223036Z.to.20100208T223035Z.sigtar.gpg
Last full backup date: Tue Feb  2 10:40:33 2010
No old backup sets found, nothing deleted.

phase 2: root:
Local and Remote metadata are synchronized, no sync needed.
Last full backup date: Mon Feb  1 21:42:18 2010
No old backup sets found, nothing deleted.

phase 3: home:
Local and Remote metadata are synchronized, no sync needed.
Last full backup date: Sat Jan 30 23:30:04 2010
--------------[ Backup Statistics ]--------------
StartTime 1265754623.26 (Tue Feb  9 23:30:23 2010)
EndTime 1265754627.92 (Tue Feb  9 23:30:27 2010)
ElapsedTime 4.66 (4.66 seconds)
SourceFiles 9511
SourceFileSize 7741867237 (7.21 GB)
NewFiles 5
NewFileSize 580995 (567 KB)
DeletedFiles 0
ChangedFiles 0
ChangedFileSize 0 (0 bytes)
ChangedDeltaSize 0 (0 bytes)
DeltaEntries 5
RawDeltaSize 568707 (555 KB)
TotalDestinationSizeChange 537748 (525 KB)
Errors 0
-------------------------------------------------

phase 4: vmail:
Local and Remote metadata are synchronized, no sync needed.
Warning, found the following remote orphaned signature files:
duplicity-new-signatures.20100203T223035Z.to.20100204T223041Z.sigtar.gpg
duplicity-new-signatures.20100204T223041Z.to.20100205T223035Z.sigtar.gpg
duplicity-new-signatures.20100205T223035Z.to.20100206T223034Z.sigtar.gpg
duplicity-new-signatures.20100206T223034Z.to.20100207T223036Z.sigtar.gpg
duplicity-new-signatures.20100207T223036Z.to.20100208T223035Z.sigtar.gpg
Last full backup date: Tue Feb  2 10:40:33 2010
--------------[ Backup Statistics ]--------------
StartTime 1265754649.88 (Tue Feb  9 23:30:49 2010)
EndTime 1265772511.00 (Wed Feb 10 04:28:30 2010)
ElapsedTime 17861.12 (4 hours 57 minutes 41.12 seconds)
SourceFiles 16113
SourceFileSize 6873923904 (6.40 GB)
NewFiles 153
NewFileSize 1390552592 (1.30 GB)
DeletedFiles 36
ChangedFiles 30
ChangedFileSize 29799302 (28.4 MB)
ChangedDeltaSize 0 (0 bytes)
DeltaEntries 219
RawDeltaSize 1400729823 (1.30 GB)
TotalDestinationSizeChange 896747237 (855 MB)
Errors 0
-------------------------------------------------

phase 5: root:
Local and Remote metadata are synchronized, no sync needed.
Last full backup date: Mon Feb  1 21:42:18 2010
--------------[ Backup Statistics ]--------------
StartTime 1265772780.61 (Wed Feb 10 04:33:00 2010)
EndTime 1265773341.48 (Wed Feb 10 04:42:21 2010)
ElapsedTime 560.87 (9 minutes 20.87 seconds)
SourceFiles 56852
SourceFileSize 1633552841 (1.52 GB)
NewFiles 108
NewFileSize 10140217 (9.67 MB)
DeletedFiles 51
ChangedFiles 525
ChangedFileSize 186850723 (178 MB)
ChangedDeltaSize 0 (0 bytes)
DeltaEntries 684
RawDeltaSize 63277577 (60.3 MB)
TotalDestinationSizeChange 32056794 (30.6 MB)
Errors 0
-------------------------------------------------

phase 6: diskfree:
Filesystem            Size  Used Avail Use% Mounted on
/dev/sdb1             917G   18G  854G   2% /home