Re: [Duplicity-talk] Simple full backup verify error

[edgar . soldin]

hey Felix,

see inline

On 24.01.2023 11:19, Felix Natter via Duplicity-talk wrote:
> hello Duplicity community,
>
> I am running a simple daily backup (in this case: just one full backup with 
> just one volume):
>
> export DUPLICITY_OPTS="--verbosity=debug --tempdir=/repos/temp 
> --archive-dir=/repos/.duplicity-cache --volsize=28000 --no-compression --encrypt-key 
> MY_KEY"
>
> duplicity $DUPLICITY_OPTS --full-if-older-than 5D /repos/dumps.backup/ 
> file:///mnt/FOO/daily/ |& tee /tmp/duplicity.log

not sure that you need debug verbosity here already

> which succeeds. Immediately afterwards, I run a verify including compare:
>
> duplicity verify $DUPLICITY_OPTS --compare-data file:///mnt/FOO/daily/ 
> /repos/dumps.backup/ |& tee /tmp/duplicity.log
>
> which fails with many of these:
>
> Registering (mktemp) temporary file 
> /repos/temp/duplicity-t168pkjr-tempdir/mktemp-d9mbcjxz-1
> Invalid data - SHA1 hash mismatch for file:
>   duplicity-full.20230124T082316Z.vol1.difftar.gpg
>   Calculated hash: ccab08e9879d9598b83945c83233a9978d954c67
>   Manifest hash: cf642123945b4f86b376815c8b8f2ce92e0f2688
>
> Registering (mktemp) temporary file 
> /repos/temp/duplicity-t168pkjr-tempdir/mktemp-gz1sto53-2
> Invalid data - SHA1 hash mismatch for file:
>   duplicity-full.20230124T082316Z.vol1.difftar.gpg
>   Calculated hash: ccab08e9879d9598b83945c83233a9978d954c67
>   Manifest hash: cf642123945b4f86b376815c8b8f2ce92e0f2688
>
> [...]

duplicity creates checksums during backup and these do not match anymore, vulgo 
they have been corrupted.

> /mnt/FOO is an NFS mounted share. The target directory and cache directory are 
> fresh
> (just deleted before the backup job). I haven't deleted the temp folder though.
> I am using the latest duplicity: 1.2.1.
>
> Here is more log output that occurs before the above:
>
> 3 file(s) exists on backend
> 3 file(s) exists in cache
> Extracting backup chains from list of files: 
> ['duplicity-full.20230124T082316Z.vol1.difftar.gpg', 
> 'duplicity-full-signatures.20230124T082316Z.sigtar.gpg', 
> 'duplicity-full.20230124T082316Z.manifest.gpg']
> File duplicity-full.20230124T082316Z.vol1.difftar.gpg is not part of a known 
> set; creating new set
> File duplicity-full-signatures.20230124T082316Z.sigtar.gpg is not part of a 
> known set; creating new set
> Ignoring file (rejected by backup set) 
> 'duplicity-full-signatures.20230124T082316Z.sigtar.gpg'
> Processing local manifest 
> b'/repos/.duplicity-cache/956a308cc4cd0300f1ebbd0e0b7865c1/duplicity-full.20230124T082316Z.manifest'
>  (4835)
> Found manifest volume 1
> Found 1 volumes in manifest
> File duplicity-full.20230124T082316Z.manifest.gpg is part of known set
> Found backup chain [Tue Jan 24 09:23:16 2023]-[Tue Jan 24 09:23:16 2023]
> Local and Remote metadata are synchronized, no sync needed.
> Last full backup date: Tue Jan 24 09:23:16 2023
> Collection Status
> -----------------
> Connecting with backend: BackendWrapper
> Archive dir: /repos/.duplicity-cache/956a308cc4cd0300f1ebbd0e0b7865c1
>
> Found 0 secondary backup chain(s).
>
> Found primary backup chain with matching signature chain:
> -------------------------
> Chain start time: Tue Jan 24 09:23:16 2023
> Chain end time: Tue Jan 24 09:23:16 2023
> Number of contained backup sets: 1
> Total number of contained volumes: 1
>   Type of backup set:                            Time:      Num volumes:
>                  Full         Tue Jan 24 09:23:16 2023                 1
> -------------------------
> No orphaned or incomplete backup sets found.
> Processing local manifest 
> b'/repos/.duplicity-cache/956a308cc4cd0300f1ebbd0e0b7865c1/duplicity-full.20230124T082316Z.manifest'
>  (4835)
> Found manifest volume 1
> Found 1 volumes in manifest
> Processing local manifest 
> b'/repos/.duplicity-cache/956a308cc4cd0300f1ebbd0e0b7865c1/duplicity-full.20230124T082316Z.manifest'
>  (4835)
> Found manifest volume 1
> Found 1 volumes in manifest
> Using temporary directory /repos/temp/duplicity-t168pkjr-tempdir
> Registering (mktemp) temporary file 
> /repos/temp/duplicity-t168pkjr-tempdir/mktemp-d9mbcjxz-1
>
> So what am I missing?

just tried locally with another key but essentially your DUPLICITY_OPTS and 
duplicity 1.2.1 verified fine.

how about you backup/verify to/from a local file:// target first (e.g. 
/tmp/test.backup/) and work your way up from there. this way you make sure 
everything is working and then you can start debugging the NFS issues.

good luck ..ede/duply.net