[rdiff-backup-users] 1.1.9 problem - out of memory

[Jakub Krzyz . ewski]

Hi all.

I set up backup using backupninja scripts and rdiff-backup.
Source is MacOsX 10.3.9 and destination is Linux.
rdiff-backup 1.1.9, librsync 0.9.7
Directory I'm going to backup has 300GB.
I got following error. Is it known bug? Is it python, librsync or 
rdiff-backup related? Is there any patch or workaround? I made some 
googling with no result.


May 19 21:24:38 Debug: /usr/bin/rdiff-backup --carbonfile 
--print-statistics --include '/Volumes/DANE/WPROST/' --exclude '/*
' / address@hidden::/var/raid2/backup/wprost-xserve/
May 21 11:52:27 Warning: malloc: vm_allocate(size=1718456320) failed 
(error code=3)
malloc[5623]: error: Can't allocate region Exception 'out of memory' 
raised of class 'exceptions.MemoryError': File 
"/var/lib/python-support/python2.4/rdiff_backup/robust.py", line 32, in 
check_common_error try: return function(*args)
File "/var/lib/python-support/python2.4/rdiff_backup/rpath.py", line 
101, in copy if rpin.isreg(): return copy_reg_file(rpin, rpout, compress)
File "/var/lib/python-support/python2.4/rdiff_backup/rpath.py", line 
128, in copy_reg_file return rpout.write_from_fileobj(rpin.open("rb"), 
compress = compress)
File "/var/lib/python-support/python2.4/rdiff_backup/rpath.py", line 
1053, in write_from_fileobj copyfileobj(fp, outfp)
File "/var/lib/python-support/python2.4/rdiff_backup/rpath.py", line 58, 
in copyfileobj inbuf = inputfp.read(blocksize)
File "/var/lib/python-support/python2.4/rdiff_backup/rpath.py", line 
1269, in read def read(self, length = -1): return self.file.read(length)
File "/var/lib/python-support/python2.4/rdiff_backup/iterfile.py", line 
118, in read if not self.addtobuffer(): break
File "/var/lib/python-support/python2.4/rdiff_backup/iterfile.py", line 
133, in addtobuffer type, data = self.iwf._get()
File "/var/lib/python-support/python2.4/rdiff_backup/iterfile.py", line 
414, in _get if not self.buf: self.buf += self.file.read()
File "/var/lib/python-support/python2.4/rdiff_backup/connection.py", 
line 516, in read return self.connection.VirtualFile.readfromid(self.id, 
length)
File "/var/lib/python-support/python2.4/rdiff_backup/connection.py", 
line 448, in __call__ return apply(self.connection.reval, (self.name,) + 
args)
File "/var/lib/python-support/python2.4/rdiff_backup/connection.py", 
line 370, in reval if isinstance(result, Exception): raise result 
Exception 'out of memory' raised of class 'exceptions.MemoryError':
File "/opt/local/lib/python2.4/site-packages/rdiff_backup/Main.py", line 
298, in error_check_Main try: Main(arglist)
File "/opt/local/lib/python2.4/site-packages/rdiff_backup/Main.py", line 
318, in Main take_action(rps)
File "/opt/local/lib/python2.4/site-packages/rdiff_backup/Main.py", line 
274, in take_action elif action == "backup": Backup(rps[0], rps[1])
File "/opt/local/lib/python2.4/site-packages/rdiff_backup/Main.py", line 
340, in Backup backup.Mirror(rpin, rpout)
File "/opt/local/lib/python2.4/site-packages/rdiff_backup/backup.py", 
line 38, in Mirror DestS.patch(dest_rpath, source_diffiter)
File 
"/opt/local/lib/python2.4/site-packages/rdiff_backup/connection.py", 
line 448, in __call__ return apply(self.connection.reval, (self.name,) + 
args)
File 
"/opt/local/lib/python2.4/site-packages/rdiff_backup/connection.py", 
line 370, in reval if isinstance(result, Exception): raise result
Traceback (most recent call last):
File "/usr/bin/rdiff-backup", line 23, in ? 
rdiff_backup.Main.error_check_Main(sys.argv[1:])
File "/opt/local/lib/python2.4/site-packages/rdiff_backup/Main.py", line 
298, in error_check_Main try: Main(arglist)
File "/opt/local/lib/python2.4/site-packages/rdiff_backup/Main.py", line 
318, in Main take_action(rps)
File "/opt/local/lib/python2.4/site-packages/rdiff_backup/Main.py", line 
274, in take_action elif action == "backup": Backup(rps[0], rps[1])
File "/opt/local/lib/python2.4/site-packages/rdiff_backup/Main.py", line 
340, in Backup backup.Mirror(rpin, rpout)
File "/opt/local/lib/python2.4/site-packages/rdiff_backup/backup.py", 
line 38, in Mirror DestS.patch(dest_rpath, source_diffiter)
File 
"/opt/local/lib/python2.4/site-packages/rdiff_backup/connection.py", 
line 448, in __call__ return apply(self.connection.reval, (self.name,) + 
args)
File 
"/opt/local/lib/python2.4/site-packages/rdiff_backup/connection.py", 
line 370, in reval if isinstance(result, Exception): raise result 
MemoryError: out of memory Fatal Error: Lost connection to the remote system