[Duplicity-tracker] [bug #21792] pipe call fails with an error OSError: [Errno 24] Too many open files

[Kevin Crowston]

Follow-up Comment #3, bug #21792 (project duplicity):

0.4.7 still has the same problem. 

It appears to die after about 123 calls to gpg, which is not quite half of
the 256 open file limit. Upping the file limit to 1024 gets farther, so it
does look like a problem with something not getting closed and using up file
handles. So, a work around is setting a higher file handle limit. 
 
Here's a slightly fuller dump. I can rerun it with -v9 and post the whole
transcript as an attachment if it'd be useful. 


.....
7-12-10T00:33:02-04:00.sigtar.gpg
/tmp/duplicity-3XcTDI-tempdir/mktemp-OC9oif-124' (attempt #1)
gpg: encrypted with 2048-bit ELG-E key, ID 9E7D67FC, created 2007-11-08
      "Kevin Crowston (Key for nanobot backups) <address@hidden>"
gpg: encrypted with 2048-bit ELG-E key, ID 9E7D67FC, created 2007-11-08
      "Kevin Crowston (Key for nanobot backups) <address@hidden>"
Generating delta - new file: Volumes
Generating delta - new file: Volumes/EXTERNAL/nanobot
Generating delta - new file: Volumes/EXTERNAL/nanobot/daily.0
Generating delta - new file: Volumes/EXTERNAL/nanobot/daily.0/mysql
Generating delta - new file: Volumes/EXTERNAL/nanobot/daily.0/mysql/gallery
Generating delta - changed file:
Volumes/EXTERNAL/nanobot/daily.0/mysql/gallery/g2_ItemAttributesMap.sql
Generating delta - changed file:
Volumes/EXTERNAL/nanobot/daily.0/mysql/gallery/taka_g2_ItemAttributesMap.sql
Generating delta - changed file:
Volumes/EXTERNAL/nanobot/daily.0/mysql/spamassassin.sql
Generating delta - changed file: private/var/backups/local.nidump
Generating delta - new file: private/var/cache/logwatch
Generating delta - new file: private/var/db
Generating delta - changed file: private/var/db/.dashboardadvisory.database
Generating delta - changed file: private/var/db/SystemEntropyCache
Generating delta - changed file: private/var/db/dashboardadvisory.database
Generating delta - changed file:
private/var/db/shadow/hash/BE1DD597-8A4A-4085-9219-E2B64FB2EF27.state
Generating delta - changed file: private/var/db/sudo/crowston/ttyp0
Generating delta - changed file: private/var/db/sudo/crowston/ttyp1
Generating delta - new file: private/var/imap
Generating delta - changed file: private/var/imap/annotations.db
Generating delta - new file: private/var/imap/db
Generating delta - changed file: private/var/imap/db/__db.001
Generating delta - changed file: private/var/imap/db/__db.002
Generating delta - changed file: private/var/imap/db/__db.003
Generating delta - changed file: private/var/imap/db/__db.004
Generating delta - changed file: private/var/imap/db/__db.005
Generating delta - changed file: private/var/imap/db/log.0000000029
Generating delta - changed file: private/var/imap/db/skipstamp
Generating delta - new file: private/var/imap/db.backup1
Generating delta - changed file: private/var/imap/db.backup1/annotations.db
Generating delta - changed file: private/var/imap/db.backup1/log.0000000029
Generating delta - changed file: private/var/imap/db.backup1/mailboxes.db
Generating delta - changed file: private/var/imap/db.backup1/quotas.db
Generating delta - new file: private/var/imap/db.backup2
Generating delta - changed file: private/var/imap/db.backup2/annotations.db
Generating delta - changed file: private/var/imap/db.backup2/log.0000000029
Generating delta - changed file: private/var/imap/db.backup2/mailboxes.db
Generating delta - changed file: private/var/imap/db.backup2/quotas.db
Generating delta - changed file: private/var/imap/deliver.db
Generating delta - changed file: private/var/imap/mailboxes.db
Generating delta - changed file: private/var/imap/quotas.db
Generating delta - changed file: private/var/imap/tls_sessions.db
Generating delta - new file: private/var/imap/user/c
Generating delta - changed file: private/var/imap/user/c/crowston.seen
Generating delta - new file: private/var/imap/user/w
Generating delta - changed file: private/var/imap/user/w/williams.seen
Generating delta - changed file: private/var/postgrey/log.0000000005
Generating delta - changed file: private/var/postgrey/postgrey.db
Generating delta - changed file: private/var/postgrey/postgrey_clients.db
Generating delta - new file: private/var/root/.gnupg
Generating delta - deleted file:
private/var/root/.gnupg/.#lk0x601410.nanobot.syr.edu.12870
Generating delta - deleted file:
private/var/root/.gnupg/.#lk0x601410.nanobot.syr.edu.13238
Generating delta - new file:
private/var/root/.gnupg/.#lk0x601730.nanobot.syr.edu.16961
Generating delta - new file:
private/var/root/.gnupg/.#lk0x601730.nanobot.syr.edu.17339
Generating delta - changed file: private/var/root/.gnupg/random_seed
Generating delta - new file: private/var/root/.ssh
Generating delta - changed file: private/var/root/.ssh/known_hosts
Generating delta - new file: private/var/root/.ssh/known_hosts~
Generating delta - new file: private/var/spamd/.spamassassin
Generating delta - changed file:
private/var/spamd/.spamassassin/auto-whitelist
Running 'scp  /tmp/duplicity-3XcTDI-tempdir/mktemp-dZ35W3-125
address@hidden:backup/nanobot-hourly/duplicity-inc.2007-12-09T23:33:02-05:00.to.2007-12-13T16:55:34-05:00.vol1.difftar.gpg'
(attempt #1)
Traceback (most recent call last):
  File "/usr/local/bin/duplicity", line 378, in <module>
    if __name__ == "__main__": main()
  File "/usr/local/bin/duplicity", line 372, in main
    else: incremental_backup(sig_chain)
  File "/usr/local/bin/duplicity", line 175, in incremental_backup
    bytes_written = write_multivol("inc", tarblock_iter, globals.backend)
  File "/usr/local/bin/duplicity", line 83, in write_multivol
    globals.gpg_profile,globals.volsize)
  File
"/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/duplicity/gpg.py",
line 193, in GPGWriteFile
    file = GPGFile(True, path.Path(filename), profile)
  File
"/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/duplicity/gpg.py",
line 100, in __init__
    'logger': self.logger_fp})
  File
"/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/GnuPGInterface.py",
line 357, in run
    create_fhs, attach_fhs)
  File
"/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/GnuPGInterface.py",
line 388, in _attach_fork_exec
    pipe = os.pipe()
OSError: [Errno 24] Too many open files
gpg: Error writing to `-': Broken pipe
gpg: handle plaintext failed: file write error
gpg: Error writing to `-': Broken pipe
gpg: handle plaintext failed: file write error
gpg: Error writing to `-': Broken pipe
gpg: handle plaintext failed: file write error
gpg: Error writing to `-': Broken pipe
gpg: handle plaintext failed: file write error
gpg: Error writing to `-': Broken pipe
gpg: handle plaintext failed: file write error
gpg: Error writing to `-': Broken pipe
gpg: handle plaintext failed: file write error
gpg: Error writing to `-': Broken pipe
gpg: handle plaintext failed: file write error
gpg: Error writing to `-': Broken pipe
gpg: handle plaintext failed: file write error
gpg: Error writing to `-': Broken pipe


    _______________________________________________________

Reply to this item at:

  <http://savannah.nongnu.org/bugs/?21792>

_______________________________________________
  Message sent via/by Savannah
  http://savannah.nongnu.org/