duplicity-talk
[Top][All Lists]
Advanced
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Duplicity-talk] Duplicity - Resource Unavailable Errors

From: Colin Ryan
Subject: Re: [Duplicity-talk] Duplicity - Resource Unavailable Errors
Date: Tue, 10 Nov 2009 09:31:43 -0500
User-agent: Thunderbird 2.0.0.23 (Windows/20090812) 
Update:
Tried this same test again and it is not always that this failure leaves the rest of the system unable to fork other processes such as "ls" or "top" etc etc. sometimes yes, sometimes no...in any even duplicity does continue to grow in excess of 200Mb put I do have physical RAM reported as free when this occurs. 

Any thoughts?

Colin Ryan wrote:
I should know the answer to this but can't seem to confirm. Clearly such errors are generally flimit/ulimit issues or RAM.
But I'm testing restore a not huge ( ~ 8 GB archive that is only 1 full and 45 days of incremental's) and get the following from the following command line.
/duplicity restore -v9 --encrypt-key=xxxxxxx --sign-key=xxxxxx --gpg-options='--default-key=xxxxxxx' --archive-dir="/opt/aa/service/nas/restores" --tempdir="/opt/aa/service/nas/restores" --ignore-errors s3+http://URL/<backup set name> "/opt/aa/service/nas/restores/office"
I'm using the -ignore-errors as I'm unpacking as a non-root user other than the one that did the backups so it was throwing "no permission" errors on what I expect was the chmod/chown stages ( I say this because the files themselves were extracted)...Anyhow... 

Version is 6.0.5 hand rolled in Debian Lenny.
When it ceases the system is unable to fork any other processes until I kill duplicity.
A already running top in another terminal shows that duplicity at this point is ~ 200 MB virtual and 11 MB resident but that I have ~ 200M RAM free (this is a fairly small footprint system). 

I believe it's out of memory for forking but is it normal.

<code>
BEGNj3-tempdir/mktemp-QYdP4e-85
DEBUG:duplicity:Removing still remembered temporary file /opt/aa/service/nas/restores/duplicity-BEGNj3-tempdir/mktemp-QYdP4e-85 Removing still remembered temporary file /opt/aa/service/nas/restores/duplicity-BEGNj3-tempdir/mktemp-aDi69N-61 DEBUG:duplicity:Removing still remembered temporary file /opt/aa/service/nas/restores/duplicity-BEGNj3-tempdir/mktemp-aDi69N-61 Removing still remembered temporary file /opt/aa/service/nas/restores/duplicity-BEGNj3-tempdir/mktemp-2YCFpS-81 DEBUG:duplicity:Removing still remembered temporary file /opt/aa/service/nas/restores/duplicity-BEGNj3-tempdir/mktemp-2YCFpS-81 Removing still remembered temporary file /opt/aa/service/nas/restores/duplicity-BEGNj3-tempdir/mktemp-CXX67i-73 DEBUG:duplicity:Removing still remembered temporary file /opt/aa/service/nas/restores/duplicity-BEGNj3-tempdir/mktemp-CXX67i-73 
Traceback (most recent call last):
 File "/usr/bin/duplicity", line 1241, in ?
   with_tempdir(main)
 File "/usr/bin/duplicity", line 1234, in with_tempdir
   fn()
 File "/usr/bin/duplicity", line 1188, in main
   restore(col_stats)
 File "/usr/bin/duplicity", line 542, in restore
   restore_get_patched_rop_iter(col_stats)):
File "/usr/lib/python2.4/site-packages/duplicity/patchdir.py", line 519, in Write_ROPaths 
   for ropath in rop_iter:
File "/usr/lib/python2.4/site-packages/duplicity/patchdir.py", line 491, in integrate_patch_iters 
   for patch_seq in collated:
File "/usr/lib/python2.4/site-packages/duplicity/patchdir.py", line 376, in yield_tuples 
   setrorps(overflow, elems)
File "/usr/lib/python2.4/site-packages/duplicity/patchdir.py", line 365, in setrorps 
   elems[i] = iter_list[i].next()
File "/usr/lib/python2.4/site-packages/duplicity/patchdir.py", line 110, in difftar2path_iter 
   tarinfo_list = [tar_iter.next()]
File "/usr/lib/python2.4/site-packages/duplicity/patchdir.py", line 326, in next 
   self.set_tarfile()
File "/usr/lib/python2.4/site-packages/duplicity/patchdir.py", line 320, in set_tarfile 
   self.current_fp = self.fileobj_iter.next()
 File "/usr/bin/duplicity", line 579, in get_fileobj_iter
   manifest.volume_info_dict[vol_num])
 File "/usr/bin/duplicity", line 603, in restore_get_enc_fileobj
   fileobj = tdp.filtered_open_with_delete("rb")
File "/usr/lib/python2.4/site-packages/duplicity/dup_temp.py", line 114, in filtered_open_with_delete 
   fh = FileobjHooked(path.DupPath.filtered_open(self, mode))
File "/usr/lib/python2.4/site-packages/duplicity/path.py", line 724, in filtered_open 
   return gpg.GPGFile(False, self, gpg_profile)
File "/usr/lib/python2.4/site-packages/duplicity/gpg.py", line 135, in __init__ 
   attach_fhs={'stdin': encrypt_path.open("rb"),
File "/usr/lib/python2.4/site-packages/duplicity/GnuPGInterface.py", line 365, in run 
   create_fhs, attach_fhs)
File "/usr/lib/python2.4/site-packages/duplicity/GnuPGInterface.py", line 407, in _attach_fork_exec 
   process.pid = os.fork()
OSError: [Errno 11] Resource temporarily unavailable

ERROR:duplicity:Traceback (most recent call last):
 File "/usr/bin/duplicity", line 1241, in ?
   with_tempdir(main)
 File "/usr/bin/duplicity", line 1234, in with_tempdir
   fn()
 File "/usr/bin/duplicity", line 1188, in main
   restore(col_stats)
 File "/usr/bin/duplicity", line 542, in restore
   restore_get_patched_rop_iter(col_stats)):
File "/usr/lib/python2.4/site-packages/duplicity/patchdir.py", line 519, in Write_ROPaths 
   for ropath in rop_iter:
File "/usr/lib/python2.4/site-packages/duplicity/patchdir.py", line 491, in integrate_patch_iters 
   for patch_seq in collated:
File "/usr/lib/python2.4/site-packages/duplicity/patchdir.py", line 376, in yield_tuples 
   setrorps(overflow, elems)
File "/usr/lib/python2.4/site-packages/duplicity/patchdir.py", line 365, in setrorps 
   elems[i] = iter_list[i].next()
File "/usr/lib/python2.4/site-packages/duplicity/patchdir.py", line 110, in difftar2path_iter 
   tarinfo_list = [tar_iter.next()]
File "/usr/lib/python2.4/site-packages/duplicity/patchdir.py", line 326, in next 
   self.set_tarfile()
File "/usr/lib/python2.4/site-packages/duplicity/patchdir.py", line 320, in set_tarfile 
   self.current_fp = self.fileobj_iter.next()
 File "/usr/bin/duplicity", line 579, in get_fileobj_iter
   manifest.volume_info_dict[vol_num])
 File "/usr/bin/duplicity", line 603, in restore_get_enc_fileobj
   fileobj = tdp.filtered_open_with_delete("rb")
File "/usr/lib/python2.4/site-packages/duplicity/dup_temp.py", line 114, in filtered_open_with_delete 
   fh = FileobjHooked(path.DupPath.filtered_open(self, mode))
File "/usr/lib/python2.4/site-packages/duplicity/path.py", line 724, in filtered_open 
   return gpg.GPGFile(False, self, gpg_profile)
File "/usr/lib/python2.4/site-packages/duplicity/gpg.py", line 135, in __init__ 
   attach_fhs={'stdin': encrypt_path.open("rb"),
File "/usr/lib/python2.4/site-packages/duplicity/GnuPGInterface.py", line 365, in run 
   create_fhs, attach_fhs)
File "/usr/lib/python2.4/site-packages/duplicity/GnuPGInterface.py", line 407, in _attach_fork_exec 
   process.pid = os.fork()
OSError: [Errno 11] Resource temporarily unavailable

Error in atexit._run_exitfuncs:
Traceback (most recent call last):
 File "atexit.py", line 24, in _run_exitfuncs
   func(*targs, **kargs)
 File "threading.py", line 634, in __exitfunc
   t.join()
 File "threading.py", line 540, in join
   self.__block.wait()
 File "threading.py", line 203, in wait
   waiter.acquire()
</code>

Cheers and Thanks

Colin


_______________________________________________
Duplicity-talk mailing list
address@hidden
http://lists.nongnu.org/mailman/listinfo/duplicity-talk
reply via email to
[Prev in Thread] Current Thread [Next in Thread]