Re: [rdiff-backup-users] hardlink-only mode?

[rdiff]

On Sun, 25 Feb 2007, Luke Scott wrote:

> On Sun, 2007-02-25 at 12:00 -0800, Eric wrote:
> > Impelementing FUSEfs to mount the backed up files as a directory tree of
> > /
> > + 2007-02-11--10:30:37/
> > |    Your files here...
> > + 2007-01-17--11:30:37/
> > |    More files here...
> > + current/
> > |    Current files here
> >
> > This would of course be a read-only mount.
>
> I don't know what the storage size impact is, because I don't know if
> rsync keeps diffs of files, or if it stores whole copies and only uses
> the rdiff algorithm for transport.

Only for transport... This is why tools like rdiff-backup exist.

> In any case, de-linking and re-creating a file in the source dir will 
> effectively double disk usage for that file.

Yes and no.  Basically, when /rdiff-fs/10-11-2007/some/file is accessed, 
it performs a
"rdiff-backup --restore-as-of 10-11-2007 /backups/some/file /tmp/file" and 
proxies the IO on /rdiff-fs/10-11-2007/some/file to /tmp/file.  When 
close() is called on the proxy'd IO, /tmp/file can be deleted.  I am 
thinking of using this method so that users who need to restore files can 
SFTP into the server, traverse to the directory they want to read and pull 
the files as if they are real files instead of reaching them rdiff-backup 
command line options (...they are generally windows monkies).  Opening a 
file would be slow since each open() invokes rdiff-backup --restore, but 
some read-ahead like inteligence might help this out.

I do like what you have below, though it performs hardlink "magic" which, 
while it works great, I try to avoid since copying /backup/ to another 
volume gets messy when you wish to preserve hardlinks (use 
e2dump/restore?).  Aside from the messy volume-to-volume transfer issues, 
it would work quite well.  I didn't realize that rsync even had a 
--dest-link option; kinda cool!

-Eric

> Here's the method I had in my head for a possible strategy, which might
> be exactly what you are thinking of:
>
> 1. Initial backup is made ENTIRELY by creating hardlinks to the source
> dir (no data is copied)
>
>  cp -al /source/dir /backup/2007-02-20
>
> 2. Orignial data is kept on a Copy-On-Write FUSE filesystem (idea here:
> http://xmailserver.org/flcow.html), meaning that any files with hardlink
> backups are de-linked in the source dir, and re-written - rather than
> modifying in place. This prevents programs from modifying the backups
> during write operations.
>
> 3. Subsequent backups are taken by using rsync --link-dest so that all
> copies of identical files are actually pointing to a single inode:
>
>  rsync -a --delete \
>    --link-dest=/backup/2007-02-20 \
>    /source/dir/ \
>    /backup/2007-02-21
>
> (There is a note on this method here:
> http://www.mikerubel.org/computers/rsync_snapshots/#Incremental)
>
> Is this what you were thinking?
>
> Luke