rdiff-backup-users
[Top][All Lists]
Advanced
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [rdiff-backup-users] CRC error

From: rdiff
Subject: Re: [rdiff-backup-users] CRC error
Date: Wed, 7 Nov 2007 21:33:13 -0800 (PST)
I'm not sure where this original post went to in the archives, but we have had the same problem. Below are 2 reposted messages on the subject from my sent-mail which discuss resolution. Hopefully they will find their way to the archive or perhaps be easier to find with your message below this one. 

-Eric


From address@hidden Mon Jan 22 13:35:34 2007 -0800

Date: Mon, 22 Jan 2007 13:35:34 -0800 (PST)
From: address@hidden
To: address@hidden
Subject: CRC check Failed
Message-ID: <address@hidden>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed
Status: O
X-Status:
X-Keywords:
X-UID: 4793



We are using 1.0.4 and it has been running stable for many months.  Every
once in a while this CRC check comes across and in the past, the only way
we knew to resolve it is to remove the rdiff-backup-data directory and
start over.  --check-destination-dir also fails with the same error.

After reading the mailing list, they suggest that one of the .gz files is
bad.  After something like this:

(http://www.mail-archive.com/address@hidden/msg01116.html)

find dest/rdiff-backup-data -type f -name \*.gz -print0 | \
        xargs -0r gzip --test

I found that the file
dest/rdiff-backup-data/mirror_metadata.2007-01-19T18:30:23-08:00.snapshot.gz
was bad.  The previous mirror_metadata file
"mirror_metadata.2007-01-19T05:52:24-08:00.snapshot.gz" was intact so I
copied the good one over the bad one and re-ran --check-destination-dir.
Even though I used an older mirror_metadata, --check-destination-dir
returned without error.

I realize that because I copied an old metadata over a more recent
backup's metadata some of the increments may be clobbered or perhaps
confused.  However, it is possible that the damaged metadata was a backup
which never really did anything in which case, no harm done;
unfortunately, this is unknown.
What kind of metadata confusion/increment errors/restore errors might come up due to this metadata replacement which we should be aware of? 

Is there a way to check the integrity of the increments?

Can rdiff-backup perform restore operations on each file with increments
and compare against some metadata md5/sha?

Is there a way to keep mirror_metadata's from getting damaged or making
the code which handles mirror_metadata more robust to failures?

Does anyone have suggestions to avoid these scenarios in the future?

Thank you for your help!!!

-Eric

================================================

From: address@hidden
To: Guenevere Prawiroatmodjo <address@hidden>
cc: address@hidden, address@hidden
Subject: Re: [rdiff-backup-users] Exception 'CRC check failed'
In-Reply-To: <address@hidden>
Message-ID: <address@hidden>
References: <address@hidden>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed
Status: O
X-Status:
X-Keywords:
X-UID: 4813

On Fri, 26 Jan 2007, Guenevere Prawiroatmodjo wrote:



Hello,
My last backup failed because of a hardware error, which probably caused 
some
corrupt files in the process. When I use --check-desination-dir on the 
backup

folder the following error msg appears:



This problem has come up many times on the mailing list, and so far there
is not a good fix for it.  DEVELOPERS: If you can think of a good fix
which could be added to --check-destination-dir, this would be wonderful!

I don't know enough about the inner workings of the rdiff-backup-data
tree, but usually this error appears when one of the .gz files was
incomplete.  Try this:

find rdiff-backup-data/ -name '*.gz" | while read a; do
        echo -n "$a:"
        gzip --test "$a"
done

This will tell you which files are bad.  In our case, we had a problem
with the mirror_metadata files.  Search the archive for specifics, but
basically I copied a good mirror_metadata over the bad mirror_metadata and
crossed my fingers.  It worked for us, though I do not know the
implications here.  After doing this surgery, you might try a restore of
the past few backup dates and make sure it is successful.  Worst case, you
rm-rf rdiff-backup-data, though you will loose your increments.  If you
come up with any good ideas, please let us know!

-Eric


On Wed, 7 Nov 2007, Kurt Yoder wrote:
Anyone? I'm starting to get worried, since my backups haven't run for several days now. Is there any way to fix the broken rdiff-backup-data directory? 

On Nov 1, 2007, at 10:26 PM, Kurt Yoder wrote:


Hello list
I recently started seeing this error on one of my backups. After two backups, it's still appearing: 


Traceback (most recent call last):
 File "/usr/bin/rdiff-backup", line 23, in ?
   rdiff_backup.Main.error_check_Main(sys.argv[1:])
File "/var/lib/python-support/python2.4/rdiff_backup/Main.py", line 295, in error_check_Main 
   try: Main(arglist)
File "/var/lib/python-support/python2.4/rdiff_backup/Main.py", line 315, in Main 
   take_action(rps)
File "/var/lib/python-support/python2.4/rdiff_backup/Main.py", line 271, in take_action 
   elif action == "backup": Backup(rps[0], rps[1])
File "/var/lib/python-support/python2.4/rdiff_backup/Main.py", line 328, in Backup 
   backup_final_init(rpout)
File "/var/lib/python-support/python2.4/rdiff_backup/Main.py", line 425, in backup_final_init 
   checkdest_if_necessary(rpout)
File "/var/lib/python-support/python2.4/rdiff_backup/Main.py", line 824, in checkdest_if_necessary 
   dest_rp.conn.regress.Regress(dest_rp)
File "/var/lib/python-support/python2.4/rdiff_backup/regress.py", line 71, in Regress 
   for rf in iterate_meta_rfs(mirror_rp, inc_rpath): ITR(rf.index, rf)
File "/var/lib/python-support/python2.4/rdiff_backup/rorpiter.py", line 281, in __call__ 
   last_branch.fast_process(*args)
File "/var/lib/python-support/python2.4/rdiff_backup/regress.py", line 267, in fast_process 
   if rf.metadata_rorp.isreg(): self.restore_orig_regfile(rf)
File "/var/lib/python-support/python2.4/rdiff_backup/regress.py", line 289, in restore_orig_regfile 
   tf.write_from_fileobj(rf.get_restore_fp())
File "/var/lib/python-support/python2.4/rdiff_backup/restore.py", line 485, in get_restore_fp 
   return robust.check_common_error(error_handler, get_fp)
File "/var/lib/python-support/python2.4/rdiff_backup/robust.py", line 32, in check_common_error 
   try: return function(*args)
File "/var/lib/python-support/python2.4/rdiff_backup/restore.py", line 465, in get_fp 
   Rdiff.write_patched_fp(current_fp, delta_fp, new_fp)
File "/var/lib/python-support/python2.4/rdiff_backup/Rdiff.py", line 73, in write_patched_fp 
   rpath.copyfileobj(librsync.PatchedFile(basis_fp, delta_fp), out_fp)
File "/var/lib/python-support/python2.4/rdiff_backup/rpath.py", line 58, in copyfileobj 
   inbuf = inputfp.read(blocksize)
File "/var/lib/python-support/python2.4/rdiff_backup/librsync.py", line 77, in read 
   self._add_to_outbuf_once()
File "/var/lib/python-support/python2.4/rdiff_backup/librsync.py", line 86, in _add_to_outbuf_once 
   if not self.infile_eof: self._add_to_inbuf()
File "/var/lib/python-support/python2.4/rdiff_backup/librsync.py", line 96, in _add_to_inbuf 
   new_in = self.infile.read(blocksize)
 File "/usr/lib/python2.4/gzip.py", line 225, in read
   self._read(readsize)
 File "/usr/lib/python2.4/gzip.py", line 290, in _read
   self._read_eof()
 File "/usr/lib/python2.4/gzip.py", line 309, in _read_eof
   raise IOError, "CRC check failed"
IOError: CRC check failed
So my backups are failing. After some quick Google searching, I turned up some discussion about the same kind of error on 1.1.2. It is here: 

http://osdir.com/ml/sysutils.backup.rdiff-backup.general/2006-03/msg00031.html
The only solution seemed to be to delete the rdiff-backup-data directory. I am now using 1.1.5 (Debian Etch, Linux 2.6.18). Was there any other method besides deleting the rdiff directory to fix this? 


Thanks!


_______________________________________________
rdiff-backup-users mailing list at address@hidden
http://lists.nongnu.org/mailman/listinfo/rdiff-backup-users
Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki 



_______________________________________________
rdiff-backup-users mailing list at address@hidden
http://lists.nongnu.org/mailman/listinfo/rdiff-backup-users
Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki
reply via email to
[Prev in Thread] Current Thread [Next in Thread]