[Duplicity-talk] Re: Duplicity 0.5.10 crashing

[Joel Carter]

Hey again.

Still having problems with my backup, not sure where to go with this
one. I have tried adding /var/spool to my excludes file but it still
looks like it's including that anyway. I figure since that's the last
thing it's working on before crashing that's a good thing to exclude
to see if that helps:

...
Comparing None and ('var', 'spool')
Generating delta - deleted file: var/spool
Comparing None and ('var', 'spool', 'anacron')
Generating delta - deleted file: var/spool/anacron
Comparing None and ('var', 'spool', 'anacron', 'cron.daily')
Generating delta - deleted file: var/spool/anacron/cron.daily
gpg: CAST5 encrypted data
gpg: encrypted with 1 passphrase
Deleting /tmp/duplicity-UuDb8d-tempdir/mktemp-ptGFwt-25
Forgetting temporary file /tmp/duplicity-UuDb8d-tempdir/mktemp-ptGFwt-25
gpg: CAST5 encrypted data
gpg: encrypted with 1 passphrase
Deleting /tmp/duplicity-UuDb8d-tempdir/mktemp-6-CMl6-5
Forgetting temporary file /tmp/duplicity-UuDb8d-tempdir/mktemp-6-CMl6-5
gpg: CAST5 encrypted data
gpg: encrypted with 1 passphrase
Deleting /tmp/duplicity-UuDb8d-tempdir/mktemp-QmrLB_-4
Forgetting temporary file /tmp/duplicity-UuDb8d-tempdir/mktemp-QmrLB_-4
Comparing None and ('var', 'spool', 'anacron', 'cron.monthly')
Generating delta - deleted file: var/spool/anacron/cron.monthly
Comparing None and ('var', 'spool', 'anacron', 'cron.weekly')
Generating delta - deleted file: var/spool/anacron/cron.weekly
Comparing None and ('var', 'spool', 'at')
Generating delta - deleted file: var/spool/at
Comparing None and ('var', 'spool', 'at', '.SEQ')
Generating delta - deleted file: var/spool/at/.SEQ
Comparing None and ('var', 'spool', 'at', 'spool')
Generating delta - deleted file: var/spool/at/spool
Comparing None and ('var', 'spool', 'clientmqueue')
Generating delta - deleted file: var/spool/clientmqueue
Comparing None and ('var', 'spool', 'clientmqueue', 'dfn1B802Q8010279')
Comparing None and ('var', 'spool', 'clientmqueue', 'dfn1E802ns005564')
Comparing None and ('var', 'spool', 'clientmqueue', 'dfn1F803NM014865')
Comparing None and ('var', 'spool', 'clientmqueue', 'dfn1G8032s026407')
Comparing None and ('var', 'spool', 'clientmqueue', 'dfn1H803LO003297')
Comparing None and ('var', 'spool', 'cron')
Generating delta - deleted file: var/spool/cron
Comparing None and ('var', 'spool', 'cron', 'root')
Generating delta - deleted file: var/spool/cron/root
gpg: CAST5 encrypted data
gpg: encrypted with 1 passphrase
Deleting /tmp/duplicity-UuDb8d-tempdir/mktemp-mjnMw_-6
Forgetting temporary file /tmp/duplicity-UuDb8d-tempdir/mktemp-mjnMw_-6
Comparing None and ('var', 'spool', 'cups')
Generating delta - deleted file: var/spool/cups
Comparing None and ('var', 'spool', 'cups', 'tmp')
Generating delta - deleted file: var/spool/cups/tmp
Comparing None and ('var', 'spool', 'lpd')
Generating delta - deleted file: var/spool/lpd
Comparing None and ('var', 'spool', 'mail')
Generating delta - deleted file: var/spool/mail
Comparing None and ('var', 'spool', 'mail', 'root')
Generating delta - deleted file: var/spool/mail/root
Comparing None and ('var', 'spool', 'mqueue')
Generating delta - deleted file: var/spool/mqueue
Removing still remembered temporary file
/tmp/duplicity-UuDb8d-tempdir/mktemp-IcfJDc-15
Removing still remembered temporary file
/tmp/duplicity-UuDb8d-tempdir/mktemp-SeWVzI-3
Removing still remembered temporary file
/tmp/duplicity-UuDb8d-tempdir/mktemp-C5fPPI-14
Removing still remembered temporary file
/tmp/duplicity-UuDb8d-tempdir/mktemp--uMUqL-12
Removing still remembered temporary file
/tmp/duplicity-UuDb8d-tempdir/mktemp-wuCOmw-10
Removing still remembered temporary file
/tmp/duplicity-UuDb8d-tempdir/mktemp-s0fz4Y-9
Removing still remembered temporary file
/tmp/duplicity-UuDb8d-tempdir/mktemp-UJjX8Z-27
Removing still remembered temporary file
/tmp/duplicity-UuDb8d-tempdir/mktemp-tQa8Z2-51
Removing still remembered temporary file
/tmp/duplicity-UuDb8d-tempdir/mktemp-dTnhYV-13
Removing still remembered temporary file
/tmp/duplicity-UuDb8d-tempdir/mktemp-cgiLEq-11
Removing still remembered temporary file
/tmp/duplicity-UuDb8d-tempdir/mktemp-cOZ6wr-7
Removing still remembered temporary file
/tmp/duplicity-UuDb8d-tempdir/mktemp-uC9MQG-21
Removing still remembered temporary file
/tmp/duplicity-UuDb8d-tempdir/mktemp-Tznw6T-17
Removing still remembered temporary file
/tmp/duplicity-UuDb8d-tempdir/mktemp-l880Gd-19
Removing still remembered temporary file
/tmp/duplicity-UuDb8d-tempdir/mktemp-7d_7DE-16
Removing still remembered temporary file
/tmp/duplicity-UuDb8d-tempdir/mktemp-El-wGp-18
Removing still remembered temporary file
/tmp/duplicity-UuDb8d-tempdir/mktemp-8GRms4-24
Removing still remembered temporary file
/tmp/duplicity-UuDb8d-tempdir/mktemp-EZ8mJZ-23
Removing still remembered temporary file
/tmp/duplicity-UuDb8d-tempdir/mktemp-ml3cHb-20
Removing still remembered temporary file
/tmp/duplicity-UuDb8d-tempdir/mktemp-Obablb-22
Removing still remembered temporary file
/tmp/duplicity-UuDb8d-tempdir/mktemp-Z4G5XL-8
Removing still remembered temporary file
/tmp/duplicity-UuDb8d-tempdir/mktemp-XyT4L0-2
Traceback (most recent call last):
  File "/usr/bin/duplicity", line 589, in ?
    with_tempdir(main)
  File "/usr/bin/duplicity", line 582, in with_tempdir
    fn()
  File "/usr/bin/duplicity", line 573, in main
    incremental_backup(sig_chain)
  File "/usr/bin/duplicity", line 276, in incremental_backup
    bytes_written = write_multivol("inc", tarblock_iter, globals.backend)
  File "/usr/bin/duplicity", line 149, in write_multivol
    globals.gpg_profile, globals.volsize)
  File "/usr/lib64/python2.4/site-packages/duplicity/gpg.py", line
241, in GPGWriteFile
    data = block_iter.next(bytes_to_go).data
  File "/usr/lib64/python2.4/site-packages/duplicity/diffdir.py", line
466, in next
    result = self.process(self.input_iter.next(), size)
  File "/usr/lib64/python2.4/site-packages/duplicity/diffdir.py", line
176, in get_delta_iter
    for new_path, sig_path in collated:
  File "/usr/lib64/python2.4/site-packages/duplicity/diffdir.py", line
261, in collate2iters
    relem2 = riter2.next()
  File "/usr/lib64/python2.4/site-packages/duplicity/diffdir.py", line
326, in combine_path_iters
    refresh_triple_list(triple_list)
  File "/usr/lib64/python2.4/site-packages/duplicity/diffdir.py", line
312, in refresh_triple_list
    new_triple = get_triple(old_triple[1])
  File "/usr/lib64/python2.4/site-packages/duplicity/diffdir.py", line
300, in get_triple
    path = path_iter_list[iter_index].next()
  File "/usr/lib64/python2.4/site-packages/duplicity/diffdir.py", line
235, in sigtar2path_iter
    sigtarobj.close()
  File "/usr/lib64/python2.4/site-packages/duplicity/dup_temp.py",
line 118, in close
    assert not self.fileobj.close()
  File "/usr/lib64/python2.4/site-packages/duplicity/gpg.py", line 162, in close
    self.gpg_process.wait()
  File "/usr/lib/python2.4/site-packages/GnuPGInterface.py", line 639, in wait
    raise IOError, "GnuPG exited non-zero, with code %d" % (e << 8)
IOError: GnuPG exited non-zero, with code 131072

So, since that's not helping, I'm thinking this should be my plan of action:

1. Rename hostname back so I can avoid using the --allow-source-mismatch
2. Run a complete new full backup on a 500Kb DSL line (not looking
forward to this one)

It really seems like a GnuPG issue since it's crashing with a non-zero.

Any hints guys?
Thanks, Joel.

On Tue, Mar 3, 2009 at 1:50 PM, Joel Carter <address@hidden> wrote:
> Hi there.
>
> I'm having some problems with an incremental backup on a server I
> administer. It seems to be crashing with something related to gnupg.
> Here is what I'm running:
>
> #export PASSPHRASE=$(cat /root/backup-passphrase) ; NAME=`hostname -s`
> ; duplicity --num-retries 10 -v9 --allow-source-mismatch
> --exclude-filelist /root/svn/$NAME/backup-excludes /
> rsync://address@hidden/backups/$NAME/ 2>&1 | tee `date
> +\%y\%m\%d`.duplicity.log
>
> Here is the output I'm seeing.
>
> <snip>
> Generating delta - new file: var/spool/mail/testuser1
> Selecting /var/spool/mqueue
> Comparing ('var', 'spool', 'mqueue') and ('var', 'spool', 'mqueue')
> Getting delta of (('var', 'spool', 'mqueue') /var/spool/mqueue dir)
> and (('var', 'spool', 'mqueue') dir)
> Generating delta - new file: var/spool/mqueue
> Selecting /var/spool/repackage
> Removing still remembered temporary file
> /tmp/duplicity-oHR0hw-tempdir/mktemp-Dv_8E8-13
> Removing still remembered temporary file
> /tmp/duplicity-oHR0hw-tempdir/mktemp-KdOHbf-35
> Removing still remembered temporary file
> /tmp/duplicity-oHR0hw-tempdir/mktemp-DnBA9l-11
> Removing still remembered temporary file
> /tmp/duplicity-oHR0hw-tempdir/mktemp-v-kGZJ-12
> Removing still remembered temporary file
> /tmp/duplicity-oHR0hw-tempdir/mktemp-IMFPgO-14
> Removing still remembered temporary file
> /tmp/duplicity-oHR0hw-tempdir/mktemp-gUnp2X-9
> Removing still remembered temporary file
> /tmp/duplicity-oHR0hw-tempdir/mktemp-VAN7Iu-18
> Removing still remembered temporary file
> /tmp/duplicity-oHR0hw-tempdir/mktemp-Se-Nuq-19
> Removing still remembered temporary file
> /tmp/duplicity-oHR0hw-tempdir/mktemp-xf8D9u-27
> Removing still remembered temporary file
> /tmp/duplicity-oHR0hw-tempdir/mktemp-w3Fm2e-24
> Removing still remembered temporary file
> /tmp/duplicity-oHR0hw-tempdir/mktemp-g4Tsa8-2
> Removing still remembered temporary file
> /tmp/duplicity-oHR0hw-tempdir/mktemp-K-f3zP-15
> Removing still remembered temporary file
> /tmp/duplicity-oHR0hw-tempdir/mktemp-GEU41Z-16
> Removing still remembered temporary file
> /tmp/duplicity-oHR0hw-tempdir/mktemp-yBr92P-20
> Removing still remembered temporary file
> /tmp/duplicity-oHR0hw-tempdir/mktemp-URaxr3-3
> Removing still remembered temporary file
> /tmp/duplicity-oHR0hw-tempdir/mktemp-liaQxF-21
> Removing still remembered temporary file
> /tmp/duplicity-oHR0hw-tempdir/mktemp-BYZtBD-10
> Removing still remembered temporary file
> /tmp/duplicity-oHR0hw-tempdir/mktemp-P2Px-I-7
> Removing still remembered temporary file
> /tmp/duplicity-oHR0hw-tempdir/mktemp-45LxYL-23
> Removing still remembered temporary file
> /tmp/duplicity-oHR0hw-tempdir/mktemp-n7_jvG-22
> Removing still remembered temporary file
> /tmp/duplicity-oHR0hw-tempdir/mktemp-DFguwJ-17
> Removing still remembered temporary file
> /tmp/duplicity-oHR0hw-tempdir/mktemp-LLJB2--8
> Traceback (most recent call last):
>  File "/usr/bin/duplicity", line 589, in ?
>    with_tempdir(main)
>  File "/usr/bin/duplicity", line 582, in with_tempdir
>    fn()
>  File "/usr/bin/duplicity", line 573, in main
>    incremental_backup(sig_chain)
>  File "/usr/bin/duplicity", line 276, in incremental_backup
>    bytes_written = write_multivol("inc", tarblock_iter, globals.backend)
>  File "/usr/bin/duplicity", line 149, in write_multivol
>    globals.gpg_profile, globals.volsize)
>  File "/usr/lib64/python2.4/site-packages/duplicity/gpg.py", line
> 241, in GPGWriteFile
>    data = block_iter.next(bytes_to_go).data
>  File "/usr/lib64/python2.4/site-packages/duplicity/diffdir.py", line
> 466, in next
>    result = self.process(self.input_iter.next(), size)
>  File "/usr/lib64/python2.4/site-packages/duplicity/diffdir.py", line
> 176, in get_delta_iter
>    for new_path, sig_path in collated:
>  File "/usr/lib64/python2.4/site-packages/duplicity/diffdir.py", line
> 261, in collate2iters
>    relem2 = riter2.next()
>  File "/usr/lib64/python2.4/site-packages/duplicity/diffdir.py", line
> 326, in combine_path_iters
>    refresh_triple_list(triple_list)
>  File "/usr/lib64/python2.4/site-packages/duplicity/diffdir.py", line
> 312, in refresh_triple_list
>    new_triple = get_triple(old_triple[1])
>  File "/usr/lib64/python2.4/site-packages/duplicity/diffdir.py", line
> 300, in get_triple
>    path = path_iter_list[iter_index].next()
>  File "/usr/lib64/python2.4/site-packages/duplicity/diffdir.py", line
> 235, in sigtar2path_iter
>    sigtarobj.close()
>  File "/usr/lib64/python2.4/site-packages/duplicity/dup_temp.py",
> line 118, in close
>    assert not self.fileobj.close()
>  File "/usr/lib64/python2.4/site-packages/duplicity/gpg.py", line 162, in 
> close
>    self.gpg_process.wait()
>  self.gpg_process.wait()
>  File "/usr/lib/python2.4/site-packages/GnuPGInterface.py", line 639, in wait
>    raise IOError, "GnuPG exited non-zero, with code %d" % (e << 8)
> IOError: GnuPG exited non-zero, with code 131072
> </snip>
>
> This is about an hour into the backup after it has successfully
> transferred the following files:
>
> <snip>
> -rw------- 1 symsolut symsolut  26202922 Mar  3 11:47
> duplicity-inc.20090223T080001Z.to.20090303T191412Z.vol1.difftar.gpg
> -rw------- 1 symsolut symsolut  26245372 Mar  3 12:04
> duplicity-inc.20090223T080001Z.to.20090303T191412Z.vol2.difftar.gpg
> -rw------- 1 symsolut symsolut  26258876 Mar  3 12:21
> duplicity-inc.20090223T080001Z.to.20090303T191412Z.vol3.difftar.gpg
> -rw------- 1 symsolut symsolut  26266635 Mar  3 12:37
> duplicity-inc.20090223T080001Z.to.20090303T191412Z.vol4.difftar.gpg
> -rw------- 1 symsolut symsolut  26253063 Mar  3 12:48
> duplicity-inc.20090223T080001Z.to.20090303T191412Z.vol5.difftar.gpg
> -rw------- 1 symsolut symsolut  26262059 Mar  3 12:59
> duplicity-inc.20090223T080001Z.to.20090303T191412Z.vol6.difftar.gpg
> -rw------- 1 symsolut symsolut  26239642 Mar  3 13:12
> duplicity-inc.20090223T080001Z.to.20090303T191412Z.vol7.difftar.gpg
> </snip>
>
> Thanks for any help you may have for me!
> Joel.
>