Re: [Duplicity-talk] question regarding cygwin and duplicity 0.6.02 (now 0.6.04)

[Mark D. Spiller]

Kenneth,

Thanks for the response, it took me a little while to get back to testing 
this out.  I'm not completely certain the patch works properly, but it was 
definitely beneficial in helping me to identify the section of code that I 
needed to look at!

From my understanding (which may be incorrect), even with the patch there is 
still a fatal error being triggered if maxopen is less than 1024?  Anyway, 
after commenting out the section:

 #if maxopen < 1024:
 #  log.FatalError(_("Max open files of %s is too low, should be >= 1024.\n"
 #             "Use 'ulimit -n 1024' or higher to correct.\n") % (maxopen,),
 #             log.ErrorCode.maxopen_too_low)

things worked.  I'm not sure at what point the number of open files becomes a 
limiting factor, but I was able to back up (and restore) a 500MB directory 
hierarchy without errors.

Perhaps the ideal would be to actually check for execution on cygwin, and, if 
so, potentially skip this check completely?  Otherwise, it's also easy to 
just comment these lines out as needed.

I'd be happy to jot up a simple FAQ response for installing on cygwin, to 
avoid future redundant questions on this, if you think that would be of 
benefit (didn't find anything like that in my initial search).

Thanks again for the help,
Mark



-------- Original Message --------
Subject: [Duplicity-talk] question regarding cygwin and duplicity 0.6.02
From: Kenneth Loafman <address@hidden>
To: Discussion of the backup program duplicity <address@hidden>
Date: Wednesday, July 29, 2009 11:56:39 AM

> Here's a patch that will fix the issue for Cygwin.  It appears that it
> did not make it in 0.5.18 after all, just 0.6.02.
>
> ...Ken
>
> Mark D. Spiller wrote:
> > Jacob,
> >
> > Thanks for the reply - as I mentioned, I'm running cygwin, and there is
> > simply no ulimit command - I assume that the system is hard-coded to 256
> > open files.
> >
> > After further investigation based on what you suggested, I'm wondering
> > if the ulimit issue is what is causing duplicity to exit, or if
> > something else is going on - that is the message that is given, but
> > perhaps that isn't the root cause, and I may be missing something else?
> >
> > I'm somewhat basing my setup on the instructions in this blog:
> > https://www.blogger.com/comment.g?blogID=5405223246853569941&postID=7504777772669563234&pli=1
> >
> >
> > I'm not using S3, though, I've been trying to backup to either my own
> > server (using ssh or scp) or to a local file, to make sure it wasn't the
> > install on the other side.  I'm including a transcript of a session
> > (with -v9) below - I created the "test" directory, but no files get
> > created there.  Essentially, it prints out the diagnostic information,
> > but nothing else happens.  Other than the ulimit message, no other error
> > messages...
> >
> > If anyone has some input, I'd really appreciate it!
> >
> > Thanks,
> > Mark
> >
> >
> >
> > bash-3.2$ env | grep TMPDIR
> > TMPDIR=/tmp
> > bash-3.2$ env | grep PASSPHRASE
> > PASSPHRASE=foo
> >
> > bash-3.2$ !10
> > duplicity full -v9 /cygdrive/c/Data/active file:///cygdrive/c/Data/test
> > Using archive dir:
> > /home/mds/.cache/duplicity/0fbc02fe8286bca7208ba1fabda8aadc
> > Using backup name: 0fbc02fe8286bca7208ba1fabda8aadc
> > Main action: full
> > Args: /bin/duplicity full -v9 /cygdrive/c/Data/active
> > file:///cygdrive/c/Data/test
> > /usr/bin/python 2.5.2 (r252:60911, Dec  2 2008, 09:26:14)
> > [GCC 3.4.4 (cygming special, gdc 0.12, using dmd 0.125)]
> > duplicity 0.5.18 (May 20, 2009)
> > CYGWIN_NT-5.1 Polar 1.5.25(0.156/4/2) 2008-06-12 19:34 i686
> > Using temporary directory /tmp/duplicity-wKVks9-tempdir
> > Registering (mkstemp) temporary file
> > /tmp/duplicity-wKVks9-tempdir/mkstemp-r5U-iA-1
> > Temp has 28929540096 available, backup will use approx 34078720.
> > Max open files of -1 is too low, should be >= 1024.
> > Use 'ulimit -n 1024' or higher to correct.
> >
> > Removing still remembered temporary file
> > /tmp/duplicity-wKVks9-tempdir/mkstemp-r5U-iA-1
> >
> >
> >
> >
> > -------- Original Message --------
> > Subject: [Duplicity-talk] question regarding cygwin and duplicity 0.6.02
> > From: Jacob Godserv <address@hidden>
> > To: Discussion of the backup program duplicity <address@hidden>
> > Date: Wednesday, July 29, 2009 6:22:25 AM
> >
> > > On Tue, Jul 28, 2009 at 4:36 PM, Mark D.
> > > Spiller<address@hidden> wrote:
> > > > Hi,
> > > >
> > > > Sorry to bother you, but I'm getting the following error from duplicity
> > > > 0.6.02 installed in cygwin:
> > > >
> > > > Max open files of -1 is too low, should be >= 1024.
> > > > Use 'ulimit -n 1024' or higher to correct.
> > > I would suggest running the suggested ulimit command with the given
> > > 1024 number or higher to work around this issue until a developer
> > > responds.
> > >
> > > I peeked at the source code, and what was actually fixed was the
> > > discovery of the max open file limit (from ulimit to
> > > resource.getrlimit). Your operating system still determines the
> > > maximum number of open files.
> > >
> > > If you are using cygwin to run duplicity, the -1 limit, which cannot
> > > be avoided, is supposedly caught and ignored by duplicity. If you are
> > > using some other Unix-based operating system, the -1 limit is very
> > > unsafe and should be corrected (as far as I know).
> >
> > _______________________________________________
> > Duplicity-talk mailing list
> > address@hidden
> > http://lists.nongnu.org/mailman/listinfo/duplicity-talk
>
>
> ------------------------------------------------------------------------
>
> _______________________________________________
> Duplicity-talk mailing list
> address@hidden
> http://lists.nongnu.org/mailman/listinfo/duplicity-talk