Re: [rdiff-backup-users] Verify times increasing

[Daniel Miller]

> > Ideology: I do the large verify every day on the remote system to  
> > make
> > sure my backup history is not becoming corrupt (e.g. due to disk
> > failure, etc.). Ideally I would like to verify the past year, but  
> > that
> > will obviously take way too long to be possible with my setup.
> >
> > Observations:
> > Despite reducing the amount of historical data that gets verified  
> > from
> > one year to three months,
>
> I think you are misunderstanding how --verify works.  If you say:
>
> rdiff-backup --verify-at-time 1Y
>
> it does not verify the last 1 years worth of backups.  It verifies a
> single backup a year ago (I believe the closest backup before that  
> exact
> time); hence the name "verify-at-time".

Yes, I do understand that. But to verify a one-year-old backup it must  
apply each set of differential data over that entire year to  
reconstruct the files as they existed one year ago. This effectively  
verifies that every backup between now and one year ago is also valid  
since any corruption in an increment younger than a year would show up  
in the reconstructed data for the one-year-old backup. The exception  
would be a file that got created within the past year. Such a file  
could be corrupt and I would not know it until it got to be a year  
old. This seems to be a weakness in the rdiff-backup verification  
method, but I do not know of a practical way to get around it.

> As the manual says, "Check all
> the data in the repository *at* the given time." (emphasis added).   
> That
> explains why you are not seeing the trend you expect.  It also means
> you're getting less verification than you thought.

No and no.

I'll address your second assertion first: I'm getting less  
verification than I thought. I maintain that it is effectively  
verifying the integrity of every backup increment between now and the  
point in time that I verify since it uses each of those increments to  
construct the point in time that I'm verifying. Please explain how my  
understanding is wrong.

Now back to your first assertion about why I am not seeing the trend I  
expect. Since I'm verifying a younger backup (only three months old  
rather than a year) it has less diff data to apply to construct that  
point in time and therefore it should take less time. But I have not  
observed a decrease. Instead the time to verify a three-month-old  
backup seems to be increasing at a constant rate.

Here's something I did not mention before: over the weekend (when  
there is enough time) I still do the --verify-at-time 1Y.  
Interestingly this verification takes about the same amount of time  
(sometimes less) than the --verify-at-time 3M which is done during the  
week. That doesn't make any sense at all to me.

Why is the time needed to verify a three-month-old backup not leveling  
off? And is there a way to bring down my verification times but still  
be sure that my backup archives are not becoming corrupt due to  
decaying storage media, etc? Is there some other method of  
verification that I could use, perhaps not even related to rdiff-backup?

~ Daniel