Re: [Duplicity-talk] frequent 'Connection reset by peer' errors

[edgar . soldin]

On 02.03.2017 11:26, Vladimir Marek via Duplicity-talk wrote:
> Hi,
> 
> First of all, thanks for duplicity!
> 
> I have compiled duplicity for the Turris Omnia router[1] which uses
> OpenWRT fork. I am making backup of ~700G of my documents (photos, hg
> repositories, documtes etc) to onedrive. All worked really well for past
> 14 days or so when suddenly I started to receive many of these
> tracebacks:
> 
> Traceback (most recent call last):
>   File "/usr/dup/usr/bin/duplicity", line 1546, in <module>
>     with_tempdir(main)
>   File "/usr/dup/usr/bin/duplicity", line 1540, in with_tempdir
>     fn()
>   File "/usr/dup/usr/bin/duplicity", line 1375, in main
>     action = commandline.ProcessCommandLine(sys.argv[1:])
>   File "/usr/dup/usr/lib/python2.7/site-packages/duplicity/commandline.py", 
> line 1126, in ProcessCommandLine
>     backup, local_pathname = set_backend(args[0], args[1])
>   File "/usr/dup/usr/lib/python2.7/site-packages/duplicity/commandline.py", 
> line 1015, in set_backend
>     globals.backend = backend.get_backend(bend)
>   File "/usr/dup/usr/lib/python2.7/site-packages/duplicity/backend.py", line 
> 223, in get_backend
>     obj = get_backend_object(url_string)
>   File "/usr/dup/usr/lib/python2.7/site-packages/duplicity/backend.py", line 
> 209, in get_backend_object
>     return factory(pu)
>   File 
> "/usr/dup/usr/lib/python2.7/site-packages/duplicity/backends/onedrivebackend.py",
>  line 91, in __init__
>     self.resolve_directory()
>   File 
> "/usr/dup/usr/lib/python2.7/site-packages/duplicity/backends/onedrivebackend.py",
>  line 189, in resolve_directory
>     files = self.get_files(object_id)
>   File 
> "/usr/dup/usr/lib/python2.7/site-packages/duplicity/backends/onedrivebackend.py",
>  line 217, in get_files
>     response = self.http_client.get(self.API_URI + path + '/files')
>   File "/usr/dup/usr/lib/python2.7/site-packages/requests/sessions.py", line 
> 501, in get
>     return self.request('GET', url, **kwargs)
>   File 
> "/usr/dup/usr/lib/python2.7/site-packages/requests_oauthlib/oauth2_session.py",
>  line 358, in request
>     headers=headers, data=data, **kwargs)
>   File "/usr/dup/usr/lib/python2.7/site-packages/requests/sessions.py", line 
> 488, in request
>     resp = self.send(prep, **send_kwargs)
>   File "/usr/dup/usr/lib/python2.7/site-packages/requests/sessions.py", line 
> 609, in send
>     r = adapter.send(request, **kwargs)
>   File "/usr/dup/usr/lib/python2.7/site-packages/requests/adapters.py", line 
> 479, in send
>     raise ConnectionError(err, request=request)
> ConnectionError: ('Connection aborted.', error(104, 'Connection reset by 
> peer'))
> 
> 
> I'm not sure why, my network connection seems to be reliable at the same
> time. The rate of the problem is so high that duplicity became virtually
> unusable to me (I have to retry connection five times before it works
> once). So I started to look in adapters.py and after short fiddling I
> changed
> 
> --- /usr/dup/usr/lib/python2.7/site-packages/requests/adapters.py.orig  
> 2017-03-02 11:15:12.782282496 +0100
> +++ /usr/dup/usr/lib/python2.7/site-packages/requests/adapters.py       
> 2017-03-02 11:15:23.003037590 +0100
> @@ -423,7 +423,7 @@
>                      assert_same_host=False,
>                      preload_content=False,
>                      decode_content=False,
> -                    retries=self.max_retries,
> +                    retries=20, # self.max_retries,
>                      timeout=timeout
>                  )
> 
> Out of pure luck that seems to help me and I don't see the problems
> anymore. I know that this is not proper bug report and not real fix
> (it's not even in duplicity code), but couldn't something to that effect
> be added to duplicity?
> 
> The self.max_retries is 0, so no retry is performed. No matter what
> --num-retries command line option says. Maybe there would be a way to
> bind --num-retries option to be set here too?
> 
> Sorry I don't know python nor I know any of the libraries involved so I
> don't try to pretend I know how the situation could be improved.
> 
> [1] https://omnia.turris.cz/en/
> 
> Thank you
> 

hi Vladimir,

can you revert your patch and try instead to patch 
"duplicity/backends/onedrivebackend.py" line 177 from

def resolve_directory(self):

to

@retry('resolve_directory', fatal=True)
def resolve_directory(self):

and see if the it now respects the --num-retries parameter?

..ede/duply.net