Re: [rdiff-backup-users] Robustness to errors during backup

[Frederik]

On 6/26/07, Charles Duffy <address@hidden> wrote:
> Frederik wrote:
> > Yes, it's normal that it fails in these cases. But it's not normal
> > that it is not able to recover from it without removing all
> > increments. For example, in case of corruption, it should ignore all
> > corrupted files, but it should not completely prevent creating new
> > backups or prevent restoring correct backups. Now it seems all
> > rdiff-backup operations are completely blocked until you manually
> > remove all increments, even of files which are not corrupted. This is
> > unacceptable for me. I had never such experiences with Bacula.
> >
> And I've never had such experiences with rdiff-backup -- and I have a
> very large number of servers using it on a nightly basis, and my backup
> server has run out of space (or had filesystem corruption) more than
> once. Are you sure that you're using it correctly?
>
> Does --check-destination-dir remove increments on you, or do you get
> some kind of error even after using --check-destination-dir?

--check-destination-dir happily crashes:
address@hidden backup]# rdiff-backup --check-destination-dir Luna
Exception 'CRC check failed' raised of class '<type 'exceptions.IOError'>':
 File "/usr/lib64/python2.5/site-packages/rdiff_backup/Main.py", line
298, in error_check_Main
   try: Main(arglist)
 File "/usr/lib64/python2.5/site-packages/rdiff_backup/Main.py", line
318, in Main
   take_action(rps)
 File "/usr/lib64/python2.5/site-packages/rdiff_backup/Main.py", line
276, in take_action
   elif action == "check-destination-dir": CheckDest(rps[0])
 File "/usr/lib64/python2.5/site-packages/rdiff_backup/Main.py", line
787, in CheckDest
   dest_rp.conn.regress.Regress(dest_rp)
 File "/usr/lib64/python2.5/site-packages/rdiff_backup/regress.py",
line 71, in Regress
   for rf in iterate_meta_rfs(mirror_rp, inc_rpath): ITR(rf.index, rf)
 File "/usr/lib64/python2.5/site-packages/rdiff_backup/regress.py",
line 196, in iterate_meta_rfs
   for raw_rf, metadata_rorp in collated:
 File "/usr/lib64/python2.5/site-packages/rdiff_backup/rorpiter.py",
line 100, in Collate2Iters
   try: relem2 = riter2.next()
 File "/usr/lib64/python2.5/site-packages/rdiff_backup/metadata.py",
line 274, in iterate
   for record in self.iterate_records():
 File "/usr/lib64/python2.5/site-packages/rdiff_backup/metadata.py",
line 283, in iterate_records
   next_pos = self.get_next_pos()
 File "/usr/lib64/python2.5/site-packages/rdiff_backup/metadata.py",
line 266, in get_next_pos
   newbuf = self.fileobj.read(self.blocksize)
 File "/usr/lib64/python2.5/gzip.py", line 227, in read
   self._read(readsize)
 File "/usr/lib64/python2.5/gzip.py", line 292, in _read
   self._read_eof()
 File "/usr/lib64/python2.5/gzip.py", line 311, in _read_eof
   raise IOError, "CRC check failed"

Traceback (most recent call last):
 File "/usr/bin/rdiff-backup", line 23, in <module>
   rdiff_backup.Main.error_check_Main(sys.argv[1:])
 File "/usr/lib64/python2.5/site-packages/rdiff_backup/Main.py", line
298, in error_check_Main
   try: Main(arglist)
 File "/usr/lib64/python2.5/site-packages/rdiff_backup/Main.py", line
318, in Main
   take_action(rps)
 File "/usr/lib64/python2.5/site-packages/rdiff_backup/Main.py", line
276, in take_action
   elif action == "check-destination-dir": CheckDest(rps[0])
 File "/usr/lib64/python2.5/site-packages/rdiff_backup/Main.py", line
787, in CheckDest
   dest_rp.conn.regress.Regress(dest_rp)
 File "/usr/lib64/python2.5/site-packages/rdiff_backup/regress.py",
line 71, in Regress
   for rf in iterate_meta_rfs(mirror_rp, inc_rpath): ITR(rf.index, rf)
 File "/usr/lib64/python2.5/site-packages/rdiff_backup/regress.py",
line 196, in iterate_meta_rfs
   for raw_rf, metadata_rorp in collated:
 File "/usr/lib64/python2.5/site-packages/rdiff_backup/rorpiter.py",
line 100, in Collate2Iters
   try: relem2 = riter2.next()
 File "/usr/lib64/python2.5/site-packages/rdiff_backup/metadata.py",
line 274, in iterate
   for record in self.iterate_records():
 File "/usr/lib64/python2.5/site-packages/rdiff_backup/metadata.py",
line 283, in iterate_records
   next_pos = self.get_next_pos()
 File "/usr/lib64/python2.5/site-packages/rdiff_backup/metadata.py",
line 266, in get_next_pos
   newbuf = self.fileobj.read(self.blocksize)
 File "/usr/lib64/python2.5/gzip.py", line 227, in read
   self._read(readsize)
 File "/usr/lib64/python2.5/gzip.py", line 292, in _read
   self._read_eof()
 File "/usr/lib64/python2.5/gzip.py", line 311, in _read_eof
   raise IOError, "CRC check failed"
IOError: CRC check failed

So I see no way to recover any of the increments.

--
Frederik