[Top][All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [Duplicity-talk] OSError: [Errno 24] Too many open files
From: |
Kenneth Loafman |
Subject: |
Re: [Duplicity-talk] OSError: [Errno 24] Too many open files |
Date: |
Mon, 11 May 2009 15:33:13 -0500 |
User-agent: |
Thunderbird 2.0.0.21 (X11/20090318) |
I cannot reproduce this on a 32bit machine at all. I'll be able to test
on a 64bit machine sometime this week and, hopefully, find the problem.
...Ken
address@hidden wrote:
> i seem to get closer to it ...
> the more incrementals (regardless if file: or ftp: backend) the more
> defunct gpg processes appear .. but dissappear again while the backup is
> still running
>
> I did
> initial backup of 3GB
> touched a new test file to have a change & did a 2nd backup
> touched a new test2 file to have a change & did a 3nd backup
> ...
>
> and since the 3rd backup the defunct gpg processes keep appearing
>
> essentially I say: Take any more than 3 incrementals in size backup and
> do a new incremental and you will see those zombie gpg processes ..
>
> while I am not sure that this is what happens on the ulimit problem I
> thought I should report these observations ... ede
>
> PS:
>
> here a 'ps xf -u user' shortly after starting the sixth run .. the
> number of gpg zombies seems to be number of sets (full + incr's) minus 2 ..
>
> 9122 pts/0 S+ 0:00 \_ /bin/bash
> /srv/www/user//release/ftplicity_1.4.2/ftplicity test backup
> 19128 pts/0 S+ 0:00 \_ /bin/bash
> /srv/www/user//release/ftplicity_1.4.2/ftplicity backup
> 19193 pts/0 S+ 0:00 \_ /bin/bash
> /srv/www/user//release/ftplicity_1.4.2/ftplicity backup
> 19194 pts/0 R+ 0:03 \_ /usr/bin/python
> /srv/www/user/_apps/duplicity-0.5.10/bin/duplicity --verbosity 4
> --encrypt-key 00000000--sign-key 00000000--gpg-options=--always-trust
> --volsize 1 --exclude-globbing-filelist /srv/www/user//.ftplicity/test/excl
> 19196 pts/0 SL+ 0:00 \_ gpg --logger-fd 4
> --passphrase-fd 8 --batch --no-tty --default-key 00000000--recipient
> 00000000--no-secmem-warning --always-trust --encrypt --sign
> 19197 pts/0 SL+ 0:00 \_ gpg --status-fd 6
> --passphrase-fd 11 --logger-fd 5 --batch --no-tty --default-key
> 00000000--no-secmem-warning --always-trust --decrypt
> 19199 pts/0 Z+ 0:00 \_ [gpg] <defunct>
> 19200 pts/0 Z+ 0:00 \_ [gpg] <defunct>
> 19201 pts/0 Z+ 0:00 \_ [gpg] <defunct>
> 19202 pts/0 Z+ 0:00 \_ [gpg] <defunct>
> 19203 pts/0 SL+ 0:00 \_ gpg --logger-fd
> 23 --passphrase-fd 28 --batch --no-tty --default-key 00000000--recipient
> 00000000--no-secmem-warning --always-trust --encrypt --sign
>
>
>> hmm .. you are right these processes are zombie (they appear in top as
>> such) .. later on the processes disappear, while duplicity is still
>> running ..
>> but I don't feel this is normal.. or is it?
>> just to be more sure, I'll try a 1MB chunk full backup of my 3GB home
>> dir .. let's see with how many zombies I'll end up ...
>>
>> interestingly my ulimit = unlimited ... seems to be default in the old
>> SUSE 10.2, btw also on my debian 5.0 box both 32bit
>> can't remember to ever have touched this setting
>>
>> ..ede
>> --
>>> These are not open processes, they are zombie processes, so no real
>>> resources taken. That simplifies it a bit. What normally causes this
>>> is the failure of a parent process to properly retrieve its exit status,
>>> but this is not the case or more people would have this problem.
>>>
>>> As far as I can tell, the only systems this is happening on are 64bit,
>>> and not even all of those. Some of my systems are 64bit and they don't
>>> show this, so I'm wondering if this could be limited to a particular
>>> version of GnuPG, or what.
>>>
>>> Edgar, from your note I could not tell, is this happening to you too?
>>>
>>> ...Ken
>>>
>>> address@hidden wrote:
>>>
>>>> just my quick observation .. a simple incr backup opens and leaves over
>>>> 20 gpg processes open. I don't think this is healthy and also can
>>>> imagine that this multiplies if the volumes get smaller (mine are
>>>> 50MB).
>>>>
>>>> regards ede
>>>>
>>>>
>>>> address@hidden:~> ps -u user xf
>>>> PID TTY STAT TIME COMMAND
>>>> 32731 ? S 0:00 sshd: address@hidden/0
>>>> 32732 pts/0 Ss 0:00 \_ -bash
>>>> 1167 pts/0 S 0:00 \_ /bin/bash /usr/local/bin/ftplicity
>>>> profile backup
>>>> 1168 pts/0 S 0:00 | \_ /bin/bash
>>>> /srv/www//release/ftplicity_1.4.2/ftplicity profile backup
>>>> 1174 pts/0 S 0:00 | \_ /bin/bash
>>>> /srv/www//release/ftplicity_1.4.2/ftplicity backup
>>>> 1239 pts/0 S 0:00 | \_ /bin/bash
>>>> /srv/www//release/ftplicity_1.4.2/ftplicity backup
>>>> 1240 pts/0 S 0:05 | \_ /usr/bin/python
>>>> /srv/www/user/_apps/duplicity-0.5.10/bin/duplicity --verbosity 4
>>>> --encrypt-key XXXXXXX --sign-key B59ECD99 --gpg-options=--always-trust
>>>> --full-if-older-than 1M --volsize 50 --exclude-globbing-filelist
>>>> /srv/www/...
>>>> 1249 pts/0 SL 0:00 | \_ gpg --logger-fd 4
>>>> --passphrase-fd 8 --batch --no-tty --default-key XXXXXXXX --recipient
>>>> XXXXXXXX --no-secmem-warning --always-trust --encrypt --sign
>>>> 1265 pts/0 SL 0:00 | \_ gpg --status-fd 6
>>>> --passphrase-fd 11 --logger-fd 5 --batch --no-tty --default-key
>>>> XXXXXXXX
>>>> --no-secmem-warning --always-trust --decrypt
>>>> 1267 pts/0 Z 0:00 | \_ [gpg] <defunct>
>>>> 1272 pts/0 Z 0:00 | \_ [gpg] <defunct>
>>>> 1275 pts/0 Z 0:00 | \_ [gpg] <defunct>
>>>> 1278 pts/0 Z 0:00 | \_ [gpg] <defunct>
>>>> 1282 pts/0 Z 0:00 | \_ [gpg] <defunct>
>>>> 1286 pts/0 Z 0:00 | \_ [gpg] <defunct>
>>>> 1289 pts/0 Z 0:00 | \_ [gpg] <defunct>
>>>> 1292 pts/0 Z 0:00 | \_ [gpg] <defunct>
>>>> 1296 pts/0 Z 0:00 | \_ [gpg] <defunct>
>>>> 1299 pts/0 Z 0:00 | \_ [gpg] <defunct>
>>>> 1302 pts/0 Z 0:00 | \_ [gpg] <defunct>
>>>> 1305 pts/0 Z 0:00 | \_ [gpg] <defunct>
>>>> 1308 pts/0 Z 0:00 | \_ [gpg] <defunct>
>>>> 1311 pts/0 Z 0:00 | \_ [gpg] <defunct>
>>>> 1314 pts/0 Z 0:00 | \_ [gpg] <defunct>
>>>> 1317 pts/0 Z 0:00 | \_ [gpg] <defunct>
>>>> 1320 pts/0 Z 0:00 | \_ [gpg] <defunct>
>>>> 1324 pts/0 Z 0:00 | \_ [gpg] <defunct>
>>>> 1327 pts/0 Z 0:00 | \_ [gpg] <defunct>
>>>> 1330 pts/0 Z 0:00 | \_ [gpg] <defunct>
>>>> 1333 pts/0 Z 0:00 | \_ [gpg] <defunct>
>>>> 1334 pts/1 Ss+ 0:00 | \_ /usr/bin/ncftp -u
>>>> ******* backup.server.de
>>>> 1335 pts/0 R+ 0:00 \_ ps -u user xf
>>>>
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> Duplicity-talk mailing list
>>>> address@hidden
>>>> http://lists.nongnu.org/mailman/listinfo/duplicity-talk
>>>>
>>>>
>>>
>>>
>>>
>>> ------------------------------------------------------------------------
>>>
>>> _______________________________________________
>>> Duplicity-talk mailing list
>>> address@hidden
>>> http://lists.nongnu.org/mailman/listinfo/duplicity-talk
>>>
>>
>>
>>
>> _______________________________________________
>> Duplicity-talk mailing list
>> address@hidden
>> http://lists.nongnu.org/mailman/listinfo/duplicity-talk
>
>
>
> _______________________________________________
> Duplicity-talk mailing list
> address@hidden
> http://lists.nongnu.org/mailman/listinfo/duplicity-talk
>
signature.asc
Description: OpenPGP digital signature
- Re: [Duplicity-talk] OSError: [Errno 24] Too many open files, (continued)
- Re: [Duplicity-talk] OSError: [Errno 24] Too many open files, Edward Jiang, 2009/05/08
- Re: [Duplicity-talk] OSError: [Errno 24] Too many open files, Kenneth Loafman, 2009/05/08
- Re: [Duplicity-talk] OSError: [Errno 24] Too many open files, Yang Zhang, 2009/05/08
- Re: [Duplicity-talk] OSError: [Errno 24] Too many open files, Kenneth Loafman, 2009/05/08
- Re: [Duplicity-talk] OSError: [Errno 24] Too many open files, Kenneth Loafman, 2009/05/08
- Re: [Duplicity-talk] OSError: [Errno 24] Too many open files, Yang Zhang, 2009/05/10
- Re: [Duplicity-talk] OSError: [Errno 24] Too many open files, edgar . soldin, 2009/05/11
- Re: [Duplicity-talk] OSError: [Errno 24] Too many open files, Kenneth Loafman, 2009/05/11
- Re: [Duplicity-talk] OSError: [Errno 24] Too many open files, edgar . soldin, 2009/05/11
- Re: [Duplicity-talk] OSError: [Errno 24] Too many open files, edgar . soldin, 2009/05/11
- Re: [Duplicity-talk] OSError: [Errno 24] Too many open files,
Kenneth Loafman <=
- Re: [Duplicity-talk] OSError: [Errno 24] Too many open files, Edgar Soldin, 2009/05/11
- Re: [Duplicity-talk] OSError: [Errno 24] Too many open files, Kenneth Loafman, 2009/05/11
- Re: [Duplicity-talk] OSError: [Errno 24] Too many open files, Edgar Soldin, 2009/05/11
- Re: [Duplicity-talk] OSError: [Errno 24] Too many open files, Kenneth Loafman, 2009/05/12
- Re: [Duplicity-talk] OSError: [Errno 24] Too many open files, edgar . soldin, 2009/05/12
- Re: [Duplicity-talk] OSError: [Errno 24] Too many open files, Kenneth Loafman, 2009/05/12
- Re: [Duplicity-talk] OSError: [Errno 24] Too many open files, edgar . soldin, 2009/05/12
- Re: [Duplicity-talk] OSError: [Errno 24] Too many open files, Yang Zhang, 2009/05/12
- Re: [Duplicity-talk] OSError: [Errno 24] Too many open files, Ian Barton, 2009/05/12
- Re: [Duplicity-talk] OSError: [Errno 24] Too many open files, Jeff Singer, 2009/05/12