duplicity-talk
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Duplicity-talk] Restore crashes in robust.py: SOLVED


From: Simon Blandford
Subject: Re: [Duplicity-talk] Restore crashes in robust.py: SOLVED
Date: Mon, 22 Dec 2008 15:54:39 +0000
User-agent: Thunderbird 2.0.0.18 (X11/20081120)

I traced the problem back and found that GPG worked on the problematic difftar.gpg file if I downloaded it and unencrypted it manually. Tar was then able to extract the files with no problems. There error GPG was throwing was "gpg: [don't know]: invalid packet (ctb=14)" and apparently this is non-fatal. There is a fix for this being rolled into GPG 2.x.
http://lists.gnupg.org/pipermail/gnupg-devel/2006-September/023180.html

I traced the point Duplicity was failing due to this back to gpg.py line 154 where self.gpg_process.wait() receives this particular error.

I copied the work-around from collections.py to trap the error and ignore it. Here is what I replaced the self.gpg_process.wait() statement with...

           try:
               self.gpg_process.wait()
           except IOError, message:
if message.args[0] != "GnuPG exited non-zero, with code 131072":
                   raise


Hope that helps.



Simon Blandford wrote:
robust.py seems to be handling an error incorrectly by passing a string to errno.errorcode instead of an error code.

There are no errors reported when making the backup. Only with the restore. It's a new, full backup.

I am unable to restore a backup from S3 without a fatal crash. I urgently need help finding a solution because I effectively have very little usable backup right now. I can accept one or two files not being restored correctly but not with the restore process itself braking without recovery. I recommend anyone else making backups to S3 to check that they can actually fully restore the data from time to time, even if the backup reported no errors.

I am using the following version of Duplicity and dependencies.
duplicity-0.5.03
librsync-0.9.7
pexpect-2.3
GnuPGInterface-0.3.2
ncftp-3.2.1
python-boto-1.6a
gnupg-1.4.5

I have re-made the backup using the above package versions and tried to restore it to an EC2 machine (so that the restore is fast and cheap).

The crash always happens at the same point in a particular backup. In the latest case at about 6GB into it out of about 50GB. The backup includes some very large webcast video media files, up to a few 100MB each.

If I look at what robust.py is trying to do I find that an "if" statement is failing because a string is passed as an error code. I break just before the offending if statement and look at the variables.

1) The variable being tested...
(Pdb) p exc
<exceptions.IOError instance at 0x1e66758>

2) The first test in the "if" statement...
(Pdb) p not isinstance(exc, EnvironmentError)
False

3) The second test in the "if" statement
(Pdb) p p errno.errorcode[exc[0]] in ['EPERM', 'ENOENT', 'EACCES', 'EBUSY', 'EEXIST', 'ENOTDIR', 'ENAMETOOLONG', 'EINTR', 'ENOTEMPTY', 'EIO', 'ETXTBSY', 'ESRCH', 'EINVAL']
*** SyntaxError: <exceptions.SyntaxError instance at 0x1e66dd0>

4) The content of exc[0]
(Pdb) p exc[0]
'GnuPG exited non-zero, with code 131072'


I have invoked pdb at the point at which duplicity was crashing in robust.py with pdb.set_trace() like so...

import pdb

def check_common_error(error_handler, function, args = ()):
   """Apply function to args, if error, run error_handler on exception

   This only catches certain exceptions which seem innocent
   enough.

   """
   # todo: import here to avoid circular dependency issue
   import duplicity.path as path

   try: return function(*args)
   #except (EnvironmentError, SkipFileException, DSRPPermError,
   #       RPathException, Rdiff.RdiffException,
   #       librsync.librsyncError, C.UnknownFileTypeError), exc:
   #   TracebackArchive.add()
except (EnvironmentError, librsync.librsyncError, path.PathException), exc:
       pdb.set_trace()
       if (not isinstance(exc, EnvironmentError) or
           (errno.errorcode[exc[0]] in
            ['EPERM', 'ENOENT', 'EACCES', 'EBUSY', 'EEXIST',
             'ENOTDIR', 'ENAMETOOLONG', 'EINTR', 'ENOTEMPTY',
             'EIO', 'ETXTBSY', 'ESRCH', 'EINVAL'])):
           #Log.exception()
           if error_handler: return error_handler(exc, *args)
       else:
           #Log.exception(1, 2)
           raise



_______________________________________________
Duplicity-talk mailing list
address@hidden
http://lists.nongnu.org/mailman/listinfo/duplicity-talk





reply via email to

[Prev in Thread] Current Thread [Next in Thread]