[rdiff-backup-users] Re: strange "Exception '[Errno 39] Directory not empty:" error

[Christopher Hawkins]

I found the issue, but hopefully there is a more graceful way to handle this 
kind of thing. My exclude list is here:
/root/snapshots
/mnt/
/var/lock/subsys/
**var/lib/heartbeat/
/sys
/proc
/dev
/tmp
**.pid
**.*$
**.swp
/root/snapshots
/u
/u_recovery

The problem is that rdiff-backup is ignoring **.pid files at my request, and in 
the directory that it cannot delete is a pid file. So it doesn't delete it and 
then blows up when it can't remove the directory. Probably there should be a 
check for a non-empty directory, and then skip gracefully if non-empty == true. 
AND/ OR, if --force is specified, then call rmtree instead of rmdir, which 
apparently is supposed to handle non empty directories by just deleting all 
their contents as well. Thoughts on this?

Chris


----- "Christopher Hawkins" <address@hidden> wrote:

> Hello,
> 
> I'm hoping that someone with more rdiff-backup / python experience can
> help me figure this out. I have a backup of my root filesystem that I
> am trying to restore. But the restore fails while trying to remove a
> non-empty directory... I have tried rdiff-backup 1.28 from Centos
> stable branch and also 1.33 built from source. Same scenario for both.
> I don't care about the data, just trying to figure out the issue with
> rdiff-backup.
> 
> The command is this:
> rdiff-backup --terminal-verbosity 9 --force
> --exclude-globbing-filelist /tmp/excludelist -r 1266948364
> /root/snapshots /
> 
> In the exclude list are typical things like /dev, /tmp, /proc, and of
> course the directory containing the snapshot. There are no NFS mounts
> involved or anything like that. All is local. What's odd is that the
> directory it is trying to remove is nothing special. I was able to
> manually remove it just fine:
> 
> address@hidden rdiff-backup-1.3.3]# rm -r /cluster/shadow/db/acdmis/run/
> rm: descend into directory `/cluster/shadow/db/acdmis/run/'? y
> rm: remove regular file
> `/cluster/shadow/db/acdmis/run//navigator.pid'? y
> rm: remove directory `/cluster/shadow/db/acdmis/run/'? y
> 
> See below, the snippet from the verbose console leading up to the
> error and then the error itself:
> 
> 
> Wed Feb 24 07:53:29 2010  Processing changed file cluster/online
> Wed Feb 24 07:53:29 2010  Processing changed file
> cluster/online/bptest
> Wed Feb 24 07:53:29 2010  Regular copying ('cluster', 'online',
> 'bptest') to /cluster/online/rdiff-backup.tmp.1577
> Wed Feb 24 07:53:29 2010  Writing file object to
> /cluster/online/rdiff-backup.tmp.1577
> Wed Feb 24 07:53:29 2010  Hash
> e5fa44f2b31c1fb553b6021e7360d07d5d91ff5e of cluster/online/bptest
> verified
> Wed Feb 24 07:53:29 2010  Copying attributes from ('cluster',
> 'online', 'bptest') to /cluster/online/rdiff-backup.tmp.1577
> Wed Feb 24 07:53:29 2010  Setting time of
> /cluster/online/rdiff-backup.tmp.1577 to 1266948306
> Wed Feb 24 07:53:29 2010  Renaming
> /cluster/online/rdiff-backup.tmp.1577 to /cluster/online/bptest
> Wed Feb 24 07:53:29 2010  Processing changed file cluster/shadow
> Wed Feb 24 07:53:29 2010  Copying attributes from ('cluster',
> 'online') to /cluster/online
> Wed Feb 24 07:53:29 2010  Setting time of /cluster/online to
> 1266947702
> Wed Feb 24 07:53:29 2010  Processing changed file cluster/shadow/db
> Wed Feb 24 07:53:29 2010  Processing changed file
> cluster/shadow/db/acdmis
> Wed Feb 24 07:53:29 2010  Regular copying ('cluster', 'shadow', 'db',
> 'acdmis') to /cluster/shadow/db/rdiff-backup.tmp.1578
> Wed Feb 24 07:53:29 2010  Processing changed file
> cluster/shadow/db/acdmis/run
> Wed Feb 24 07:53:29 2010  Regular copying ('cluster', 'shadow', 'db',
> 'acdmis', 'run') to /cluster/shadow/db/acdmis/rdiff-backup.tmp.1579
> Wed Feb 24 07:53:29 2010  Processing changed file
> cluster/shadow/db/data
> Wed Feb 24 07:53:29 2010  Removing directory
> /cluster/shadow/db/acdmis/run
> Wed Feb 24 07:53:29 2010  Exception '[Errno 39] Directory not empty:
> '/cluster/shadow/db/acdmis/run'' raised of class 'exceptions.OSError':
>   File "/usr/lib/python2.4/site-packages/rdiff_backup/Main.py", line
> 306, in error_check_Main
>     try: Main(arglist)
>   File "/usr/lib/python2.4/site-packages/rdiff_backup/Main.py", line
> 326, in Main
>     take_action(rps)
>   File "/usr/lib/python2.4/site-packages/rdiff_backup/Main.py", line
> 292, in take_action
>     elif action == "restore-as-of": Restore(rps[0], rps[1], 1)
>   File "/usr/lib/python2.4/site-packages/rdiff_backup/Main.py", line
> 589, in Restore
>     inc_rpath, dest_rp, time)
>   File "/usr/lib/python2.4/site-packages/rdiff_backup/restore.py",
> line 37, in Restore
>     TargetS.patch(target, diff_iter)
>   File "/usr/lib/python2.4/site-packages/rdiff_backup/restore.py",
> line 293, in patch
>     ITR(diff.index, diff)
>   File "/usr/lib/python2.4/site-packages/rdiff_backup/rorpiter.py",
> line 277, in __call__
>     if self.finish_branches(index) is None:
>   File "/usr/lib/python2.4/site-packages/rdiff_backup/rorpiter.py",
> line 229, in finish_branches
>     to_be_finished.end_process()
>   File "/usr/lib/python2.4/site-packages/rdiff_backup/restore.py",
> line 672, in end_process
>     self.base_rp.rmdir()
>   File "/usr/lib/python2.4/site-packages/rdiff_backup/rpath.py", line
> 993, in rmdir
>     self.conn.os.rmdir(self.path)
> 
> Traceback (most recent call last):
>   File "/usr/bin/rdiff-backup", line 30, in ?
>     rdiff_backup.Main.error_check_Main(sys.argv[1:])
>   File "/usr/lib/python2.4/site-packages/rdiff_backup/Main.py", line
> 306, in error_check_Main
>     try: Main(arglist)
>   File "/usr/lib/python2.4/site-packages/rdiff_backup/Main.py", line
> 326, in Main
>     take_action(rps)
>   File "/usr/lib/python2.4/site-packages/rdiff_backup/Main.py", line
> 292, in take_action
>     elif action == "restore-as-of": Restore(rps[0], rps[1], 1)
>   File "/usr/lib/python2.4/site-packages/rdiff_backup/Main.py", line
> 589, in Restore
>     inc_rpath, dest_rp, time)
>   File "/usr/lib/python2.4/site-packages/rdiff_backup/restore.py",
> line 37, in Restore
>     TargetS.patch(target, diff_iter)
>   File "/usr/lib/python2.4/site-packages/rdiff_backup/restore.py",
> line 293, in patch
>     ITR(diff.index, diff)
>   File "/usr/lib/python2.4/site-packages/rdiff_backup/rorpiter.py",
> line 277, in __call__
>     if self.finish_branches(index) is None:
>   File "/usr/lib/python2.4/site-packages/rdiff_backup/rorpiter.py",
> line 229, in finish_branches
>     to_be_finished.end_process()
>   File "/usr/lib/python2.4/site-packages/rdiff_backup/restore.py",
> line 672, in end_process
>     self.base_rp.rmdir()
>   File "/usr/lib/python2.4/site-packages/rdiff_backup/rpath.py", line
> 993, in rmdir
>     self.conn.os.rmdir(self.path)
> OSError: [Errno 39] Directory not empty:
> '/cluster/shadow/db/acdmis/run'
> 
> address@hidden rdiff-backup-1.3.3]# ls /cluster/shadow/db/acdmis/run/
> navigator.pid
> 
> So python is correct, the directory is not empty. But why does cause
> an error? Looking for any thoughts / suggestions!
> 
> Thanks,
> Chris