[Duplicity-talk] frequent 'Connection reset by peer' errors

[Vladimir Marek]

Hi,

First of all, thanks for duplicity!

I have compiled duplicity for the Turris Omnia router[1] which uses
OpenWRT fork. I am making backup of ~700G of my documents (photos, hg
repositories, documtes etc) to onedrive. All worked really well for past
14 days or so when suddenly I started to receive many of these
tracebacks:

Traceback (most recent call last):
  File "/usr/dup/usr/bin/duplicity", line 1546, in <module>
    with_tempdir(main)
  File "/usr/dup/usr/bin/duplicity", line 1540, in with_tempdir
    fn()
  File "/usr/dup/usr/bin/duplicity", line 1375, in main
    action = commandline.ProcessCommandLine(sys.argv[1:])
  File "/usr/dup/usr/lib/python2.7/site-packages/duplicity/commandline.py", 
line 1126, in ProcessCommandLine
    backup, local_pathname = set_backend(args[0], args[1])
  File "/usr/dup/usr/lib/python2.7/site-packages/duplicity/commandline.py", 
line 1015, in set_backend
    globals.backend = backend.get_backend(bend)
  File "/usr/dup/usr/lib/python2.7/site-packages/duplicity/backend.py", line 
223, in get_backend
    obj = get_backend_object(url_string)
  File "/usr/dup/usr/lib/python2.7/site-packages/duplicity/backend.py", line 
209, in get_backend_object
    return factory(pu)
  File 
"/usr/dup/usr/lib/python2.7/site-packages/duplicity/backends/onedrivebackend.py",
 line 91, in __init__
    self.resolve_directory()
  File 
"/usr/dup/usr/lib/python2.7/site-packages/duplicity/backends/onedrivebackend.py",
 line 189, in resolve_directory
    files = self.get_files(object_id)
  File 
"/usr/dup/usr/lib/python2.7/site-packages/duplicity/backends/onedrivebackend.py",
 line 217, in get_files
    response = self.http_client.get(self.API_URI + path + '/files')
  File "/usr/dup/usr/lib/python2.7/site-packages/requests/sessions.py", line 
501, in get
    return self.request('GET', url, **kwargs)
  File 
"/usr/dup/usr/lib/python2.7/site-packages/requests_oauthlib/oauth2_session.py", 
line 358, in request
    headers=headers, data=data, **kwargs)
  File "/usr/dup/usr/lib/python2.7/site-packages/requests/sessions.py", line 
488, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/dup/usr/lib/python2.7/site-packages/requests/sessions.py", line 
609, in send
    r = adapter.send(request, **kwargs)
  File "/usr/dup/usr/lib/python2.7/site-packages/requests/adapters.py", line 
479, in send
    raise ConnectionError(err, request=request)
ConnectionError: ('Connection aborted.', error(104, 'Connection reset by peer'))


I'm not sure why, my network connection seems to be reliable at the same
time. The rate of the problem is so high that duplicity became virtually
unusable to me (I have to retry connection five times before it works
once). So I started to look in adapters.py and after short fiddling I
changed

--- /usr/dup/usr/lib/python2.7/site-packages/requests/adapters.py.orig  
2017-03-02 11:15:12.782282496 +0100
+++ /usr/dup/usr/lib/python2.7/site-packages/requests/adapters.py       
2017-03-02 11:15:23.003037590 +0100
@@ -423,7 +423,7 @@
                     assert_same_host=False,
                     preload_content=False,
                     decode_content=False,
-                    retries=self.max_retries,
+                    retries=20, # self.max_retries,
                     timeout=timeout
                 )

Out of pure luck that seems to help me and I don't see the problems
anymore. I know that this is not proper bug report and not real fix
(it's not even in duplicity code), but couldn't something to that effect
be added to duplicity?

The self.max_retries is 0, so no retry is performed. No matter what
--num-retries command line option says. Maybe there would be a way to
bind --num-retries option to be set here too?

Sorry I don't know python nor I know any of the libraries involved so I
don't try to pretend I know how the situation could be improved.

[1] https://omnia.turris.cz/en/

Thank you
-- 
        Vlad