[rdiff-backup-users] Re: New feature: --verify-full

[Daniel Miller]

I have implemented a --verify-full option... <snip>

Here's the patch for 1.2.8

OK, after testing this on a large backup over the weekend I have to say this is definitely not production ready yet. I'm seeing some very strange errors which appear to be related to metadata writing problems. The metadata writer seems to be garbling data as it writes. I haven't found the source of that bug yet... I didn't even think I did anything that would have affected metadata writing. I've included an example of the corruption at the end of this message. The issue is only appearing about 80% through a 100+ GB backup, which is even more frustrating (why does it start at that point?)

In general, I'm disappointed with the quality of the code in rdiff-backup. It is very fragile. Some things that contribute to this: Frequent use of global variables makes it hard to know what will be impacted when editing the code. Non-standard use of classes (static.MakeClass) makes subclassing to reduce duplication difficult. There's a lot of code duplication with very small differences, which makes it difficult to add enhancements and/or refactor the code. One specific example of this is GzipFile write buffering. This should be done in one place (probably rpath.GzipFile). Instead, it's done in at least two places, and each one is implemented a little bit differently.

I'm getting discouraged with this modification, which I thought should have been fairly straightforward. Especially now that I'm seeing big-time corruption in places I didn't even think I'd touched. I would be grateful if someone would review my patch and tell me if I've overlooked something simple.

~ Daniel


Example of metadata corruption:

File ODUsers/daniel/Code/PyOE/temp/ReportLab-2.3/src/rl_addons/renderPM/libart_lgpl/NEWS
18ffa6cb324e4df2pae3dda71266e9c7b76756c2ator:00000000|type:00000000|location:0,0|flags3
 94a474555c301ffc3fa21c2d5beb 42554
 ResourceFork None
e 32a33c31becg318fa5d7460ae2a5039a209d1e 1253799928
 Uid 1041
 Unad 1041Came.se.inPyOE/temp/ReportLab-76 staff
 Permissions 420
File ODUsers/daniel/Code/PyOE/temp/ReportLab-2.3/src/rl_addons/renderPM/libart_lgpl/README.CVS
6ad5343d4b3f2bd538dd6a4dc252aa3ourca89c2ator:00000000|type:00000000|location:0,0|flags3
 94a474555c301ffc3fa21c2d5beb 42554
 ResourceFork None
e 32a33c31becg318fa5d7460ae2a5039a209d1e 1253799928
 Uid 1041
 UnatVS
e daniel
 Gid 20
 Gname staf8/raff
 Permissions 420