[Duplicity-talk] bug or not: --archive-dir and full backup

[Thomas Seliger]

Hi *,

I am setting up a duplicity based remote backup at the moment, so first of all: 
many thanks for this great piece of software ;)

While playing around with duplicity 0.4.3 I found the following behaviour:

I use a wrapper script (derived from ftplicity) to backup files, I use scp/sftp 
as transport backend. For a full backup the following duplicity command is 
executed:

/usr/bin/duplicity --ssh-options -oPort=10022 
-oIdentityFile=/usr/local/scplicity/conf/id_dsa \
--archive-dir /usr/local/scplicity/conf \
--encrypt-key 01234567 --sign-key 01234567 \
--verbosity 4 --include-globbing-filelist /usr/local/scplicity/conf/include \
--exclude-globbing-filelist /usr/local/scplicity/conf/exclude --full / \
scp://address@hidden//usr/local/scplicity/backupdir/

After this duplicity-full-signatures.(full_backup_timestamp).sigtar.gz and 
duplicity-full.(full_backup_timestamp).manifest are located in the local 
archive dir. On the remote side are two files: duplicity-full.*.manifest.gpg 
and duplicity-full.*.vol1.difftar.gpg

Now I am running a incremental backup with:

/usr/bin/duplicity --ssh-options -oPort=10022 
-oIdentityFile=/usr/local/scplicity/conf/id_dsa \
--archive-dir /usr/local/scplicity/conf \
--encrypt-key 01234567 --sign-key 01234567 \
--verbosity 4 --include-globbing-filelist /usr/local/scplicity/conf/include \
--exclude-globbing-filelist /usr/local/scplicity/conf/exclude / \
scp://address@hidden//usr/local/scplicity/backupdir/

Everything seems fine until I run an incremental backup again, which runs ok 
but gives me the following message:

Warning, found the following orphaned signature files:
duplicity-new-signatures.(timestamp_of_first_incremental).sigtar.gpg

It seemed to me that the remote signature chain is broken, because of a missing 
duplicity-full-signatures.(full_backup_timestamp).sigtar.gpg on the remote 
side. Putting an gpg encrypted local 
duplicity-full-signatures.(full_backup_timestamp).sigtar on the remote side 
fixed this.

Duplicity now displays "Warning, found unnecessary signature chain(s)", which I 
assume is correct behaviour.

I dipped into duplicity source and found the following full backup code:

def full_backup(col_stats):
        """Do full backup of directory to backend, using archive_dir"""
        sig_outfp = get_sig_fileobj("full-sig")
        tarblock_iter = diffdir.DirFull_WriteSig(globals.select, sig_outfp)
        bytes_written = write_multivol("full", tarblock_iter, globals.backend)
        sig_outfp.close()
        col_stats.set_values(sig_chain_warning = None).cleanup_signatures()
        print_statistics(diffdir.stats, bytes_written)

I commented out "col_stats.set_values(sig_chain_warning = 
None).cleanup_signatures()" as it seemed to remove the remote signature file on a 
full backup. Running full backup and incremental backups now gave me a complete signature 
chains on the local and the remote end.

Why is the remote signature removed after a full backup? Is it intended od 
maybe a bug?
Will my changes make something go horribly wrong because I messed it all up ;)?

Peace,
Tom