[Top][All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [Duplicity-talk] some usage questions regarding full/incremental bac
From: |
Kenneth Loafman |
Subject: |
Re: [Duplicity-talk] some usage questions regarding full/incremental backups |
Date: |
Sat, 03 Jan 2009 07:35:32 -0600 |
User-agent: |
Thunderbird 2.0.0.18 (X11/20081125) |
Mathijs Kwik wrote:
> Hi,
>
> I currently use rdiff-backup to backup a few servers.
> They all backup to 1 central backup-machine.
> That machine thus has backup for all servers, including history.
> This is all unencrypted, but this isn't an issue since the machine is
> in the office and is secured by itself (disks are encrypted).
> After all machines have backed-up to this central location, it uses
> regular rsync to upload the archive to an off-site machine.
> (I don't use rdiff-backup for this, since I don't need history of history :)
> The off-site machine is in a fully secured, trusted environment.
>
> Now,
> I would like to change the off-site machine to a regular cheap
> backup-hosting service (current location is _very_ expensive).
> So I need encryption since I can't trust the backup-hosting provider.
I would stay with the local server for quick recovery when needed and
not go directly to the backup-hosting service.
> So I found duplicity, it looks perfect, just the way I need it.
>
> One thing that isn't clear to me is the full/incremental stuff.
> Currently, using rdiff-backup, I can remove all stuff that's older than a
> month.
> I saw duplicity can do this too, but the man-page mentions it won't
> delete old stuff that has newer (incremental) stuff depending on it.
> Maybe rdiff-backup has the same issue, didn't look into that yet.
> So, if I only use incremental backups, the archive will fill-up more
> and more even when cleaning old stuff.
Yes, that is the way it works. It is not a good practice to only do
incremental backups, especially with files that change a lot.
> So I need to use full backups every now and then.
> Will this mean everything gets transfered fully?
Only on a full backup.
> Or is there some intelligent algorithm that justs 'merges' the
> increments into a full backup?
No, but the idea has been put out before.
> Since our total backup is close to 1TB, I don't feel like transferring
> everything every month or so.
> Also, storage space might be an issue.
> Say I want to keep a month of backups,
> I would do a full backup every 2 weeks and clean up all but the last 2
> full's right after that.
> Does this mean that (right after doing the full backup, before
> cleaning up), there are 3 full copies?
> needing 3TB+ storage feels a bit much when the 'real' data is 'only' 1TB.
> Am I miscalculating here? Or is there some smart strategy I can try?
> All servers combined is about 1TB. daily changes are between 500Mb and 1Gb.
I usually do the cleanup of old backups prior to backing up so I don't
run into the situation you describe. As long as you leave the latest
full backup, you should be in good shape.
> Like I said, maybe rdiff-backup has the same issue, but I can't
> remember a full/incremental option on it.
> As far as I remember rdiff-backup chose a reverse-incremental
> strategy, so the most current backup is always full, and every
> previous backup is decremented from that.
This is similar to the reverse-delta method that CVS uses. It works and
has no need of full/incremental backups. Purging can be done at the end
of the chain.
> So I really hope some smart merging strategy can be used when doing full.
> Otherwise a reverse/decremental strategy would be very helpful to save
> space / bandwidth.
There is no merging. The space is saved because the incremental backups
only backup the changed portions of the files. It's possible to do the
merging (its done during a restore anyway), but not yet.
If you google "rdiff-backup encryption" there are a fair number of
articles on how to do this. Perhaps that is all you need? I think
duplicity will do what you need, but maybe not what you want.
...Ken
signature.asc
Description: OpenPGP digital signature