Re: [Duplicity-talk] Restore crashes in robust.py: SOLVED

[Kenneth Loafman]

Thanks!  I'll get it into the next release.

...Ken

Simon Blandford wrote:
> I traced the problem back and found that GPG worked on the problematic
> difftar.gpg file if I downloaded it and unencrypted it manually. Tar was
> then able to extract the files with no problems. There error GPG was
> throwing was "gpg: [don't know]: invalid packet (ctb=14)" and apparently
> this is non-fatal. There is a fix for this being rolled into GPG 2.x.
> http://lists.gnupg.org/pipermail/gnupg-devel/2006-September/023180.html
> 
> I traced the point Duplicity was failing due to this back to gpg.py line
> 154 where self.gpg_process.wait() receives this particular error.
> 
> I copied the work-around from collections.py to trap the error and
> ignore it. Here is what I replaced the self.gpg_process.wait() statement
> with...
> 
>            try:
>                self.gpg_process.wait()
>            except IOError, message:
>                if message.args[0] != "GnuPG exited non-zero, with code
> 131072":
>                    raise
> 
> 
> Hope that helps.
> 
> 
> 
> Simon Blandford wrote:
>> robust.py seems to be handling an error incorrectly by passing a
>> string to errno.errorcode instead of an error code.
>>
>> There are no errors reported when making the backup. Only with the
>> restore. It's a new, full backup.
>>
>> I am unable to restore a backup from S3 without a fatal crash. I
>> urgently need help finding a solution because I effectively have very
>> little usable backup right now. I can accept one or two files not
>> being restored correctly but not with the restore process itself
>> braking without recovery. I recommend anyone else making backups to S3
>> to check that they can actually fully restore the data from time to
>> time, even if the backup reported no errors.
>>
>> I am using the following version of Duplicity and dependencies.
>> duplicity-0.5.03
>> librsync-0.9.7
>> pexpect-2.3
>> GnuPGInterface-0.3.2
>> ncftp-3.2.1
>> python-boto-1.6a
>> gnupg-1.4.5
>>
>> I have re-made the backup using the above package versions and tried
>> to restore it to an EC2 machine (so that the restore is fast and cheap).
>>
>> The crash always happens at the same point in a particular backup. In
>> the latest case at about 6GB into it out of about 50GB. The backup
>> includes some very large webcast video media files, up to a few 100MB
>> each.
>>
>> If I look at what robust.py is trying to do I find that an "if"
>> statement is failing because a string is passed as an error code. I
>> break just before the offending if statement and look at the variables.
>>
>> 1) The variable being tested...
>> (Pdb) p exc
>> <exceptions.IOError instance at 0x1e66758>
>>
>> 2) The first test in the "if" statement...
>> (Pdb) p not isinstance(exc, EnvironmentError)
>> False
>>
>> 3) The second test in the "if" statement
>> (Pdb) p p errno.errorcode[exc[0]] in ['EPERM', 'ENOENT', 'EACCES',
>> 'EBUSY', 'EEXIST', 'ENOTDIR', 'ENAMETOOLONG', 'EINTR', 'ENOTEMPTY',
>> 'EIO', 'ETXTBSY', 'ESRCH', 'EINVAL']
>> *** SyntaxError: <exceptions.SyntaxError instance at 0x1e66dd0>
>>
>> 4) The content of exc[0]
>> (Pdb) p exc[0]
>> 'GnuPG exited non-zero, with code 131072'
>>
>>
>> I have invoked pdb at the point at which duplicity was crashing in
>> robust.py with pdb.set_trace() like so...
>>
>> import pdb
>>
>> def check_common_error(error_handler, function, args = ()):
>>    """Apply function to args, if error, run error_handler on exception
>>
>>    This only catches certain exceptions which seem innocent
>>    enough.
>>
>>    """
>>    # todo: import here to avoid circular dependency issue
>>    import duplicity.path as path
>>
>>    try: return function(*args)
>>    #except (EnvironmentError, SkipFileException, DSRPPermError,
>>    #       RPathException, Rdiff.RdiffException,
>>    #       librsync.librsyncError, C.UnknownFileTypeError), exc:
>>    #   TracebackArchive.add()
>>    except (EnvironmentError, librsync.librsyncError,
>> path.PathException), exc:
>>        pdb.set_trace()
>>        if (not isinstance(exc, EnvironmentError) or
>>            (errno.errorcode[exc[0]] in
>>             ['EPERM', 'ENOENT', 'EACCES', 'EBUSY', 'EEXIST',
>>              'ENOTDIR', 'ENAMETOOLONG', 'EINTR', 'ENOTEMPTY',
>>              'EIO', 'ETXTBSY', 'ESRCH', 'EINVAL'])):
>>            #Log.exception()
>>            if error_handler: return error_handler(exc, *args)
>>        else:
>>            #Log.exception(1, 2)
>>            raise
>>
>>
>>
>> _______________________________________________
>> Duplicity-talk mailing list
>> address@hidden
>> http://lists.nongnu.org/mailman/listinfo/duplicity-talk
> 
> 
> 
> _______________________________________________
> Duplicity-talk mailing list
> address@hidden
> http://lists.nongnu.org/mailman/listinfo/duplicity-talk
>

signature.asc
Description: OpenPGP digital signature