Re: [rdiff-backup-users] multiple archive hardlink space saving?

[Alexander 'Leo' Bergolth]

Hi!

On 08/18/2009 10:17 PM, Joshua Jensen wrote:
> Are there any provisions for hard link savings for backing up many
> similar machines?
> 
> I ask because I have 2000 (yes, two thousand) machines to back up.  At
> least 90% of data is the same... basic Linux filesystems, mostly / and
> /var and /usr partitions.
> 
> I wonder if it would be worthwhile to add a feature to rdiff-backup
> that would look to hardlink files that different rdiff-backup archives
> have in common?

You should be able to use a hardlinking utility like freedup,
hardlink.py, hardlink or hardlink++ on the destination filespace after
having backed up with rdiff-backup.

Some time ago, I did some investigations concerning hardlinked files in
rdiff-backup repositories, rdiff-backup does handle those files very
well. See
http://thread.gmane.org/gmane.comp.sysutils.backup.rdiff-backup.general/4350
for details.

If you want to backup thousands of machines, you will need to break down
the hardlinking jobs to smaller pieces, since the hardlinking utilities
needs to keep the metadata of every possible candidate for hardlinking
in memory. Selecting the candidates via "find" and passing them to any
hardlink utility that supports it might be an option.

Keep in mind that you have to use rdiff-backups restore feature in order
to properly restore all file attributes on hardlinked files.

Hope that helps,
Cheers,
--leo
-- 
e-mail   ::: Leo.Bergolth (at) wu.ac.at
fax      ::: +43-1-31336-906050
location ::: IT-Services | Vienna University of Economics | Austria