[rdiff-backup-users] recreation of hard links? (was: sshfs: workaround and test results)

[Andreas Olsson]

On Tuesday 08 January 2008 00:30:18 Andreas Olsson wrote:
> On Monday 07 January 2008 20:33:13 Andrew Ferguson wrote:
> > To my knowledge, they are stored in the metadata as hard links, and are
> > recreated as such during the restore.
>
> You are right. I must have been tired. Now when I redo my testcases the
> hard links are recreated just the way they are supposed to.

I must admit, I am confused. I do remember remembering doing those tests and I 
do rember remembering that they showed that those hard links came back. What I 
don't remember is the actual tests themselves or how I came to the conclusion 
I came to. I should probably start sleeping more at night. Anyway, I have no 
idea to know whatever I was right or not.

The reason I'm confused is that there is no way that I can now recreate hard 
links from rdiff-backups metadata. No matter if I backup hard links to an 
sshfs-mount, an encfs-paranoid-mount or a cifs-mount I never get any hard 
links back. All I get is two identifical files. The last couple of days I've 
performed these kinds of tests with diffrent versions of rdiff-backup and 
python on diffrent computers (all running linux, ext3).

Here we have the basic test case

address@hidden:~/hardlinks# rdiff-backup --version
rdiff-backup 1.1.15
address@hidden:~/hardlinks# python -V
Python 2.5.1
address@hidden:~# sshfs address@hidden:/root/loop/ /mnt/sshfs/ -o 
workaround=rename
address@hidden's password: 
address@hidden:~# cd /root/hardlinks/
address@hidden:~/hardlinks# echo "first line" > alfa
address@hidden:~/hardlinks# ln alfa beta
address@hidden:~/hardlinks# echo "second line" >> beta
address@hidden:~/hardlinks# diff alfa beta
address@hidden:~/hardlinks# rdiff-backup /root/hardlinks/ /mnt/sshfs/
Warning: hard linking not supported by filesystem 
at /mnt/sshfs/rdiff-backup-data
address@hidden:~/hardlinks# rm *
address@hidden:~/hardlinks# rdiff-backup -r now /mnt/sshfs/ /root/hardlinks/
address@hidden:~/hardlinks# echo "third line" >> beta
address@hidden:~/hardlinks# diff alfa beta
2a3
> third line
address@hidden:~/hardlinks#

Well, no hard links recreated there. As I mentioned previously I've tried this 
with other versions of rdiff-backup as well as python and gotten the same 
results. The same happens both against a CIFS (samba) mount and against a 
encfs-paranoid mount, so I don't think it has anything with sshfs in 
particular to do.

I am wondering if the problem is that rdiff-backup don't save any data on the 
hard links or if there is an error in recreating them. When I've run 
rdiff-backup with a higher verbosity the only mention of hard links I've seen 
is the initial test on the destinations capabilities. Perhaps this implies 
rdiff-backup isn't even aware of hard links being involed in the backup?

Here is the content of the mirror_metadata from the example above.

address@hidden:~# 
zcat 
/mnt/sshfs/rdiff-backup-data/mirror_metadata.2008-01-10T21\:59\:37+01\:00.snapshot.gz
 
File .
  Type dir
  ModTime 1199998522
  Uid 0
  Uname root
  Gid 0
  Gname root
  Permissions 493
File alfa
  Type reg
  Size 23
  SHA1Digest 16ec9d6615be3620ae619e559cc5baa8721967bb
  ModTime 1199998541
  Uid 0
  Uname root
  Gid 0
  Gname root
  Permissions 420
File beta
  Type reg
  Size 23
  SHA1Digest 16ec9d6615be3620ae619e559cc5baa8721967bb
  ModTime 1199998541
  Uid 0
  Uname root
  Gid 0
  Gname root
  Permissions 420
address@hidden:~# 

On the other hand, when doing the same backup against a local folder, 
supporting hard links nativly, I also get the following info in each files 
metadata 

  NumHardLinks 2
  Inode 4538655
  DeviceLoc 65025

Is there anyone who has managed to get hard links recreated who can share some 
info on it? What versions are running? What is the metadata supposed to look 
like? Is there any output regarding to the links showed when running on 
higher verbosity? 

By the way, here is the rdiff-backup from earlier with a bit higher verbosity.

address@hidden:~/hardlinks# rdiff-backup -v7 /root/hardlinks/ /mnt/sshfs/
Using rdiff-backup version 1.1.15
Making directory /mnt/sshfs/rdiff-backup-data
Unable to import module xattr.
Extended attributes not supported on filesystem at /root/hardlinks
Unable to import module posix1e from pylibacl package.
ACLs not supported on filesystem at /root/hardlinks
escape_dos_devices not required by filesystem at /root/hardlinks
-----------------------------------------------------------------
Detected abilities for source (read only) file system:
  Access control lists                         Off
  Extended attributes                          Off
  Case sensitivity                             On
  Escape DOS devices                           Off
  Mac OS X style resource forks                Off
  Mac OS X Finder information                  Off
-----------------------------------------------------------------
Making directory /mnt/sshfs/rdiff-backup-data/rdiff-backup.tmp.0
Touching /mnt/sshfs/rdiff-backup-data/rdiff-backup.tmp.0/5-_ a.snapshot.gz
Deleting /mnt/sshfs/rdiff-backup-data/rdiff-backup.tmp.0/5-_ a.snapshot.gz
Touching /mnt/sshfs/rdiff-backup-data/rdiff-backup.tmp.0/uniᄉ
Deleting /mnt/sshfs/rdiff-backup-data/rdiff-backup.tmp.0/uniᄉ
Touching /mnt/sshfs/rdiff-backup-data/rdiff-backup.tmp.0/:\"
Deleting /mnt/sshfs/rdiff-backup-data/rdiff-backup.tmp.0/:\"
Touching /mnt/sshfs/rdiff-backup-data/rdiff-backup.tmp.0/A
Deleting /mnt/sshfs/rdiff-backup-data/rdiff-backup.tmp.0/A
Touching /mnt/sshfs/rdiff-backup-data/rdiff-backup.tmp.0/foo
Deleting /mnt/sshfs/rdiff-backup-data/rdiff-backup.tmp.0/foo
Making directory /mnt/sshfs/rdiff-backup-data/rdiff-backup.tmp.0/hl
Touching /mnt/sshfs/rdiff-backup-data/rdiff-backup.tmp.0/hardlinked_file1
Hard 
linking /mnt/sshfs/rdiff-backup-data/rdiff-backup.tmp.0/hl/hardlinked_file2 
to /mnt/sshfs/rdiff-backup-data/rdiff-backup.tmp.0/hardlinked_file1
Warning: hard linking not supported by filesystem 
at /mnt/sshfs/rdiff-backup-data
Unable to import module xattr.
Extended attributes not supported on filesystem 
at /mnt/sshfs/rdiff-backup-data/rdiff-backup.tmp.0
Unable to import module posix1e from pylibacl package.
ACLs not supported on filesystem 
at /mnt/sshfs/rdiff-backup-data/rdiff-backup.tmp.0
Touching /mnt/sshfs/rdiff-backup-data/rdiff-backup.tmp.0/dir_inc_check
Deleting /mnt/sshfs/rdiff-backup-data/rdiff-backup.tmp.0/dir_inc_check
Touching /mnt/sshfs/rdiff-backup-data/rdiff-backup.tmp.0/regfile
Deleting /mnt/sshfs/rdiff-backup-data/rdiff-backup.tmp.0/regfile
Touching /mnt/sshfs/rdiff-backup-data/rdiff-backup.tmp.0/high_perms
Deleting /mnt/sshfs/rdiff-backup-data/rdiff-backup.tmp.0/high_perms
Touching /mnt/sshfs/rdiff-backup-data/rdiff-backup.tmp.0/symlinked_file1
Deleting /mnt/sshfs/rdiff-backup-data/rdiff-backup.tmp.0/symlinked_file2
Deleting /mnt/sshfs/rdiff-backup-data/rdiff-backup.tmp.0/symlinked_file1
escape_dos_devices not required by filesystem 
at /mnt/sshfs/rdiff-backup-data/rdiff-backup.tmp.0
Deleting /mnt/sshfs/rdiff-backup-data/rdiff-backup.tmp.0
Removing directory /mnt/sshfs/rdiff-backup-data/rdiff-backup.tmp.0
-----------------------------------------------------------------
Detected abilities for destination (read/write) file system:
  Ownership changing                           On
  Hard linking                                 Off
  fsync() directories                          On
  Directory inc permissions                    Off
  High-bit permissions                         On
  Symlink permissions                          Off
  Extended filenames                           On
  Windows reserved filenames                   Off
  Access control lists                         Off
  Extended attributes                          Off
  Case sensitivity                             On
  Escape DOS devices                           Off
  Mac OS X style resource forks                Off
  Mac OS X Finder information                  Off
-----------------------------------------------------------------
Backup: must_escape_dos_devices = 0
Making directory /mnt/sshfs/rdiff-backup-data/increments
Starting mirror /root/hardlinks to /mnt/sshfs
Processing changed file .
Processing changed file alfa
Regular copying ('alfa',) to /mnt/sshfs/rdiff-backup.tmp.1
Writing file object to /mnt/sshfs/rdiff-backup.tmp.1
Copying attributes from ('alfa',) to /mnt/sshfs/rdiff-backup.tmp.1
Setting time of /mnt/sshfs/rdiff-backup.tmp.1 to 1200000898
Renaming /mnt/sshfs/rdiff-backup.tmp.1 to /mnt/sshfs/alfa
Processing changed file beta
Regular copying ('beta',) to /mnt/sshfs/rdiff-backup.tmp.2
Writing file object to /mnt/sshfs/rdiff-backup.tmp.2
Copying attributes from ('beta',) to /mnt/sshfs/rdiff-backup.tmp.2
Setting time of /mnt/sshfs/rdiff-backup.tmp.2 to 1200000898
Renaming /mnt/sshfs/rdiff-backup.tmp.2 to /mnt/sshfs/beta
Copying attributes from () to /mnt/sshfs
Setting time of /mnt/sshfs to 1200000892
Writing mirror 
marker 
/mnt/sshfs/rdiff-backup-data/current_mirror.2008-01-10T22:45:07+01:00.data
Cleaning up
Touching /mnt/sshfs/rdiff-backup-data/error_log.2008-01-10T22:45:07+01:00.data
address@hidden:~/hardlinks# 
address@hidden:~/hardlinks# rdiff-backup -v7 -r now /mnt/sshfs/ /root/hardlinks/
Using rdiff-backup version 1.1.15
Using mirror root directory /mnt/sshfs
Unable to import module xattr.
Extended attributes not supported on filesystem 
at /mnt/sshfs/rdiff-backup-data
Unable to import module posix1e from pylibacl package.
ACLs not supported on filesystem at /mnt/sshfs/rdiff-backup-data
escape_dos_devices not required by filesystem at /mnt/sshfs/rdiff-backup-data
-----------------------------------------------------------------
Detected abilities for rdiff-backup repository (read only) file system:
  Access control lists                         Off
  Extended attributes                          Off
  Case sensitivity                             On
  Escape DOS devices                           Off
  Mac OS X style resource forks                Off
  Mac OS X Finder information                  Off
-----------------------------------------------------------------
Making directory /root/hardlinks/rdiff-backup.tmp.0
Touching /root/hardlinks/rdiff-backup.tmp.0/5-_ a.snapshot.gz
Deleting /root/hardlinks/rdiff-backup.tmp.0/5-_ a.snapshot.gz
Touching /root/hardlinks/rdiff-backup.tmp.0/uniᄉ
Deleting /root/hardlinks/rdiff-backup.tmp.0/uniᄉ
Touching /root/hardlinks/rdiff-backup.tmp.0/:\"
Deleting /root/hardlinks/rdiff-backup.tmp.0/:\"
Touching /root/hardlinks/rdiff-backup.tmp.0/A
Deleting /root/hardlinks/rdiff-backup.tmp.0/A
Touching /root/hardlinks/rdiff-backup.tmp.0/foo
Deleting /root/hardlinks/rdiff-backup.tmp.0/foo
Making directory /root/hardlinks/rdiff-backup.tmp.0/hl
Touching /root/hardlinks/rdiff-backup.tmp.0/hardlinked_file1
Hard linking /root/hardlinks/rdiff-backup.tmp.0/hl/hardlinked_file2 
to /root/hardlinks/rdiff-backup.tmp.0/hardlinked_file1
Unable to import module xattr.
Extended attributes not supported on filesystem 
at /root/hardlinks/rdiff-backup.tmp.0
Unable to import module posix1e from pylibacl package.
ACLs not supported on filesystem at /root/hardlinks/rdiff-backup.tmp.0
Touching /root/hardlinks/rdiff-backup.tmp.0/dir_inc_check
Deleting /root/hardlinks/rdiff-backup.tmp.0/dir_inc_check
Touching /root/hardlinks/rdiff-backup.tmp.0/regfile
Deleting /root/hardlinks/rdiff-backup.tmp.0/regfile
Touching /root/hardlinks/rdiff-backup.tmp.0/high_perms
Deleting /root/hardlinks/rdiff-backup.tmp.0/high_perms
Touching /root/hardlinks/rdiff-backup.tmp.0/symlinked_file1
Deleting /root/hardlinks/rdiff-backup.tmp.0/symlinked_file2
Deleting /root/hardlinks/rdiff-backup.tmp.0/symlinked_file1
escape_dos_devices not required by filesystem 
at /root/hardlinks/rdiff-backup.tmp.0
Deleting /root/hardlinks/rdiff-backup.tmp.0
Removing directory /root/hardlinks/rdiff-backup.tmp.0
-----------------------------------------------------------------
Detected abilities for restore target (read/write) file system:
  Ownership changing                           On
  Hard linking                                 On
  fsync() directories                          On
  Directory inc permissions                    On
  High-bit permissions                         On
  Symlink permissions                          Off
  Extended filenames                           On
  Windows reserved filenames                   Off
  Access control lists                         Off
  Extended attributes                          Off
  Case sensitivity                             On
  Escape DOS devices                           Off
  Mac OS X style resource forks                Off
  Mac OS X Finder information                  Off
-----------------------------------------------------------------
Restore: must_escape_dos_devices = 0
Starting restore of /mnt/sshfs to /root/hardlinks as it was as of Thu Jan 10 
22:46:30 2008.
Processing changed file .
Processing changed file alfa
Regular copying ('alfa',) to /root/hardlinks/rdiff-backup.tmp.1
Writing file object to /root/hardlinks/rdiff-backup.tmp.1
Hash 16ec9d6615be3620ae619e559cc5baa8721967bb of alfa verified
Copying attributes from ('alfa',) to /root/hardlinks/rdiff-backup.tmp.1
Setting time of /root/hardlinks/rdiff-backup.tmp.1 to 1200000898
Renaming /root/hardlinks/rdiff-backup.tmp.1 to /root/hardlinks/alfa
Processing changed file beta
Regular copying ('beta',) to /root/hardlinks/rdiff-backup.tmp.2
Writing file object to /root/hardlinks/rdiff-backup.tmp.2
Hash 16ec9d6615be3620ae619e559cc5baa8721967bb of beta verified
Copying attributes from ('beta',) to /root/hardlinks/rdiff-backup.tmp.2
Setting time of /root/hardlinks/rdiff-backup.tmp.2 to 1200000898
Renaming /root/hardlinks/rdiff-backup.tmp.2 to /root/hardlinks/beta
Copying attributes from () to /root/hardlinks
Setting time of /root/hardlinks to 1200000892
Restore finished
Cleaning up
address@hidden:~/hardlinks# 

Well, if anyone could spread any light on this I'd be grateful.

-- 
Andreas Olsson
http://www.andreasolsson.se/

signature.asc
Description: This is a digitally signed message part.