[Duplicity-talk] error: [Errno 32] Broken pipe / error: [Errno 104] Connection reset by peer

[Azfar Hashmi]

Below is my log. Note that if I upload another directory (comparatively
small size) then there is no issue. I tried diffrent volsize but no
luck. Other backups are also uploading fine from same host.

Using archive dir: /root/.cache/duplicity/e8a6ae68238f99c3dba6b96f7bbde570
Using backup name: e8a6ae68238f99c3dba6b96f7bbde570
Import of duplicity.backends.ftpbackend Succeeded
Import of duplicity.backends.sshbackend Succeeded
Import of duplicity.backends.cloudfilesbackend Succeeded
Import of duplicity.backends.imapbackend Succeeded
Import of duplicity.backends.localbackend Succeeded
Import of duplicity.backends.rsyncbackend Succeeded
Import of duplicity.backends.giobackend Failed: No module named gio
Import of duplicity.backends.hsibackend Succeeded
Import of duplicity.backends.tahoebackend Succeeded
Import of duplicity.backends.webdavbackend Succeeded
Import of duplicity.backends.botobackend Succeeded
Main action: full
================================================================================
duplicity 0.6.08b (March 11, 2010)
Args: /usr/bin/duplicity -v9 --volsize=5
--gpg-options=--compress-algo=bzip2 --s3-european-buckets
--s3-use-new-style full /mnt/backups/sftp-server/rsnapshot/daily.0/
s3+http://sftpdaily0
Linux backup 2.6.32-5-amd64 #1 SMP Mon Oct 3 03:59:20 UTC 2011 x86_64
/usr/bin/python 2.6.6 (r266:84292, Dec 26 2010, 22:31:48)
[GCC 4.4.5]
================================================================================
Using temporary directory /tmp/duplicity-lQBH7W-tempdir
Registering (mkstemp) temporary file
/tmp/duplicity-lQBH7W-tempdir/mkstemp-zOK1qS-1
Temp has 4102397952 available, backup will use approx 6815744.
Listing s3+http://sftpdaily0
Synchronizing remote metadata to local cache...
Deleting local
/root/.cache/duplicity/e8a6ae68238f99c3dba6b96f7bbde570/duplicity-full-signatures.20120215T121132Z.sigtar.gz
(not authoritative at backend).
Deleting local
/root/.cache/duplicity/e8a6ae68238f99c3dba6b96f7bbde570/duplicity-full.20120215T121132Z.manifest
(not authoritative at backend).
Listing s3+http://sftpdaily0
0 files exist on backend
0 files exist in cache
Extracting backup chains from list of files: []
Last full backup date: none
Collection Status
-----------------
Connecting with backend: BotoBackend
Archive dir: /root/.cache/duplicity/e8a6ae68238f99c3dba6b96f7bbde570

Found 0 secondary backup chains.
No backup chains with active signatures found
No orphaned or incomplete backup sets found.
Using temporary directory
/root/.cache/duplicity/e8a6ae68238f99c3dba6b96f7bbde570/duplicity-0TuNDh-tempdir
Registering (mktemp) temporary file
/root/.cache/duplicity/e8a6ae68238f99c3dba6b96f7bbde570/duplicity-0TuNDh-tempdir/mktemp-VnDfXo-1
Using temporary directory
/root/.cache/duplicity/e8a6ae68238f99c3dba6b96f7bbde570/duplicity-ouCKJY-tempdir
Registering (mktemp) temporary file
/root/.cache/duplicity/e8a6ae68238f99c3dba6b96f7bbde570/duplicity-ouCKJY-tempdir/mktemp-ox9X0N-1
AsyncScheduler: instantiating at concurrency 0
Registering (mktemp) temporary file
/tmp/duplicity-lQBH7W-tempdir/mktemp-g2BWLw-2
Selecting /mnt/backups/sftp-server/rsnapshot/daily.0
Comparing () and None
Getting delta of (() /mnt/backups/sftp-server/rsnapshot/daily.0 dir) and
None
A .
Selecting /mnt/backups/sftp-server/rsnapshot/daily.0/localhost
Comparing ('localhost',) and None
Getting delta of (('localhost',)
/mnt/backups/sftp-server/rsnapshot/daily.0/localhost dir) and None
A localhost
Selecting /mnt/backups/sftp-server/rsnapshot/daily.0/localhost/boot
Comparing ('localhost', 'boot') and None
Getting delta of (('localhost', 'boot')
/mnt/backups/sftp-server/rsnapshot/daily.0/localhost/boot dir) and None
A localhost/boot
Selecting
/mnt/backups/sftp-server/rsnapshot/daily.0/localhost/boot/System.map-2.6.32-5-amd64
Comparing ('localhost', 'boot', 'System.map-2.6.32-5-amd64') and None
Getting delta of (('localhost', 'boot', 'System.map-2.6.32-5-amd64')
/mnt/backups/sftp-server/rsnapshot/daily.0/localhost/boot/System.map-2.6.32-5-amd64
reg) and None
A localhost/boot/System.map-2.6.32-5-amd64
Selecting
/mnt/backups/sftp-server/rsnapshot/daily.0/localhost/boot/config-2.6.32-5-amd64
Comparing ('localhost', 'boot', 'config-2.6.32-5-amd64') and None
Getting delta of (('localhost', 'boot', 'config-2.6.32-5-amd64')
/mnt/backups/sftp-server/rsnapshot/daily.0/localhost/boot/config-2.6.32-5-amd64
reg) and None
A localhost/boot/config-2.6.32-5-amd64
Selecting
/mnt/backups/sftp-server/rsnapshot/daily.0/localhost/boot/extlinux.conf
Comparing ('localhost', 'boot', 'extlinux.conf') and None
Getting delta of (('localhost', 'boot', 'extlinux.conf')
/mnt/backups/sftp-server/rsnapshot/daily.0/localhost/boot/extlinux.conf
reg) and None
A localhost/boot/extlinux.conf
Selecting
/mnt/backups/sftp-server/rsnapshot/daily.0/localhost/boot/initrd.img-2.6.32-5-amd64
Comparing ('localhost', 'boot', 'initrd.img-2.6.32-5-amd64') and None
Getting delta of (('localhost', 'boot', 'initrd.img-2.6.32-5-amd64')
/mnt/backups/sftp-server/rsnapshot/daily.0/localhost/boot/initrd.img-2.6.32-5-amd64
reg) and None
A localhost/boot/initrd.img-2.6.32-5-amd64
Removing still remembered temporary file
/root/.cache/duplicity/e8a6ae68238f99c3dba6b96f7bbde570/duplicity-0TuNDh-tempdir/mktemp-VnDfXo-1
Cleanup of temporary file
/root/.cache/duplicity/e8a6ae68238f99c3dba6b96f7bbde570/duplicity-0TuNDh-tempdir/mktemp-VnDfXo-1
failed
Removing still remembered temporary file
/root/.cache/duplicity/e8a6ae68238f99c3dba6b96f7bbde570/duplicity-ouCKJY-tempdir/mktemp-ox9X0N-1
Cleanup of temporary file
/root/.cache/duplicity/e8a6ae68238f99c3dba6b96f7bbde570/duplicity-ouCKJY-tempdir/mktemp-ox9X0N-1
failed
AsyncScheduler: running task synchronously (asynchronicity disabled)
Uploading
s3+http://sftpdaily0/duplicity-full.20120215T125153Z.vol1.difftar.gpg
Upload
's3+http://sftpdaily0/duplicity-full.20120215T125153Z.vol1.difftar.gpg'
failed (attempt #1, reason: error: [Errno 32] Broken pipe)
Backtrace of previous error: Traceback (innermost last):
  File
"/usr/lib/python2.6/dist-packages/duplicity/backends/botobackend.py",
line 162, in put
    key.set_contents_from_filename(source_path.name, {'Content-Type':
'application/octet-stream'})
  File "/usr/lib/pymodules/python2.6/boto/s3/key.py", line 498, in
set_contents_from_filename
    self.set_contents_from_file(fp, headers, replace, cb, num_cb, policy)
  File "/usr/lib/pymodules/python2.6/boto/s3/key.py", line 455, in
set_contents_from_file
    self.send_file(fp, headers, cb, num_cb)
  File "/usr/lib/pymodules/python2.6/boto/s3/key.py", line 367, in send_file
    self.name, headers, sender=sender)
  File "/usr/lib/pymodules/python2.6/boto/s3/connection.py", line 342,
in make_request
    data, host, auth_path, sender)
  File "/usr/lib/pymodules/python2.6/boto/connection.py", line 459, in
make_request
    return self._mexe(method, path, data, headers, host, sender)
  File "/usr/lib/pymodules/python2.6/boto/connection.py", line 437, in _mexe
    raise e
 error: [Errno 32] Broken pipe

Uploading
s3+http://sftpdaily0/duplicity-full.20120215T125153Z.vol1.difftar.gpg
Upload
's3+http://sftpdaily0/duplicity-full.20120215T125153Z.vol1.difftar.gpg'
failed (attempt #2, reason: error: [Errno 32] Broken pipe)
Backtrace of previous error: Traceback (innermost last):
  File
"/usr/lib/python2.6/dist-packages/duplicity/backends/botobackend.py",
line 162, in put
    key.set_contents_from_filename(source_path.name, {'Content-Type':
'application/octet-stream'})
  File "/usr/lib/pymodules/python2.6/boto/s3/key.py", line 498, in
set_contents_from_filename
    self.set_contents_from_file(fp, headers, replace, cb, num_cb, policy)
  File "/usr/lib/pymodules/python2.6/boto/s3/key.py", line 455, in
set_contents_from_file
    self.send_file(fp, headers, cb, num_cb)
  File "/usr/lib/pymodules/python2.6/boto/s3/key.py", line 367, in send_file
    self.name, headers, sender=sender)
  File "/usr/lib/pymodules/python2.6/boto/s3/connection.py", line 342,
in make_request
    data, host, auth_path, sender)
  File "/usr/lib/pymodules/python2.6/boto/connection.py", line 459, in
make_request
    return self._mexe(method, path, data, headers, host, sender)
  File "/usr/lib/pymodules/python2.6/boto/connection.py", line 437, in _mexe
    raise e
 error: [Errno 32] Broken pipe

Uploading
s3+http://sftpdaily0/duplicity-full.20120215T125153Z.vol1.difftar.gpg
Upload
's3+http://sftpdaily0/duplicity-full.20120215T125153Z.vol1.difftar.gpg'
failed (attempt #3, reason: error: [Errno 32] Broken pipe)
Backtrace of previous error: Traceback (innermost last):
  File
"/usr/lib/python2.6/dist-packages/duplicity/backends/botobackend.py",
line 162, in put
    key.set_contents_from_filename(source_path.name, {'Content-Type':
'application/octet-stream'})
  File "/usr/lib/pymodules/python2.6/boto/s3/key.py", line 498, in
set_contents_from_filename
    self.set_contents_from_file(fp, headers, replace, cb, num_cb, policy)
  File "/usr/lib/pymodules/python2.6/boto/s3/key.py", line 455, in
set_contents_from_file
    self.send_file(fp, headers, cb, num_cb)
  File "/usr/lib/pymodules/python2.6/boto/s3/key.py", line 367, in send_file
    self.name, headers, sender=sender)
  File "/usr/lib/pymodules/python2.6/boto/s3/connection.py", line 342,
in make_request
    data, host, auth_path, sender)
  File "/usr/lib/pymodules/python2.6/boto/connection.py", line 459, in
make_request
    return self._mexe(method, path, data, headers, host, sender)
  File "/usr/lib/pymodules/python2.6/boto/connection.py", line 437, in _mexe
    raise e
 error: [Errno 32] Broken pipe

Uploading
s3+http://sftpdaily0/duplicity-full.20120215T125153Z.vol1.difftar.gpg
Upload
's3+http://sftpdaily0/duplicity-full.20120215T125153Z.vol1.difftar.gpg'
failed (attempt #4, reason: error: [Errno 32] Broken pipe)
Backtrace of previous error: Traceback (innermost last):
  File
"/usr/lib/python2.6/dist-packages/duplicity/backends/botobackend.py",
line 162, in put
    key.set_contents_from_filename(source_path.name, {'Content-Type':
'application/octet-stream'})
  File "/usr/lib/pymodules/python2.6/boto/s3/key.py", line 498, in
set_contents_from_filename
    self.set_contents_from_file(fp, headers, replace, cb, num_cb, policy)
  File "/usr/lib/pymodules/python2.6/boto/s3/key.py", line 455, in
set_contents_from_file
    self.send_file(fp, headers, cb, num_cb)
  File "/usr/lib/pymodules/python2.6/boto/s3/key.py", line 367, in send_file
    self.name, headers, sender=sender)
  File "/usr/lib/pymodules/python2.6/boto/s3/connection.py", line 342,
in make_request
    data, host, auth_path, sender)
  File "/usr/lib/pymodules/python2.6/boto/connection.py", line 459, in
make_request
    return self._mexe(method, path, data, headers, host, sender)
  File "/usr/lib/pymodules/python2.6/boto/connection.py", line 437, in _mexe
    raise e
 error: [Errno 32] Broken pipe

Uploading
s3+http://sftpdaily0/duplicity-full.20120215T125153Z.vol1.difftar.gpg
Upload
's3+http://sftpdaily0/duplicity-full.20120215T125153Z.vol1.difftar.gpg'
failed (attempt #5, reason: error: [Errno 104] Connection reset by peer)
Backtrace of previous error: Traceback (innermost last):
  File
"/usr/lib/python2.6/dist-packages/duplicity/backends/botobackend.py",
line 162, in put
    key.set_contents_from_filename(source_path.name, {'Content-Type':
'application/octet-stream'})
  File "/usr/lib/pymodules/python2.6/boto/s3/key.py", line 498, in
set_contents_from_filename
    self.set_contents_from_file(fp, headers, replace, cb, num_cb, policy)
  File "/usr/lib/pymodules/python2.6/boto/s3/key.py", line 455, in
set_contents_from_file
    self.send_file(fp, headers, cb, num_cb)
  File "/usr/lib/pymodules/python2.6/boto/s3/key.py", line 367, in send_file
    self.name, headers, sender=sender)
  File "/usr/lib/pymodules/python2.6/boto/s3/connection.py", line 342,
in make_request
    data, host, auth_path, sender)
  File "/usr/lib/pymodules/python2.6/boto/connection.py", line 459, in
make_request
    return self._mexe(method, path, data, headers, host, sender)
  File "/usr/lib/pymodules/python2.6/boto/connection.py", line 437, in _mexe
    raise e
 error: [Errno 104] Connection reset by peer

Giving up trying to upload
s3+http://sftpdaily0/duplicity-full.20120215T125153Z.vol1.difftar.gpg
after 5 attempts
Removing still remembered temporary file
/tmp/duplicity-lQBH7W-tempdir/mktemp-g2BWLw-2
Removing still remembered temporary file
/tmp/duplicity-lQBH7W-tempdir/mkstemp-zOK1qS-1
Backend error detail: Traceback (most recent call last):
  File "/usr/bin/duplicity", line 1251, in <module>
    with_tempdir(main)
  File "/usr/bin/duplicity", line 1244, in with_tempdir
    fn()
  File "/usr/bin/duplicity", line 1217, in main
    full_backup(col_stats)
  File "/usr/bin/duplicity", line 416, in full_backup
    globals.backend)
  File "/usr/bin/duplicity", line 315, in write_multivol
    (tdp, dest_filename)))
  File "/usr/lib/python2.6/dist-packages/duplicity/asyncscheduler.py",
line 145, in schedule_task
    return self.__run_synchronously(fn, params)
  File "/usr/lib/python2.6/dist-packages/duplicity/asyncscheduler.py",
line 171, in __run_synchronously
    ret = fn(*params)
  File "/usr/bin/duplicity", line 314, in <lambda>
    async_waiters.append(io_scheduler.schedule_task(lambda tdp,
dest_filename: put(tdp, dest_filename),
  File "/usr/bin/duplicity", line 240, in put
    backend.put(tdp, dest_filename)
  File
"/usr/lib/python2.6/dist-packages/duplicity/backends/botobackend.py",
line 174, in put
    raise BackendException("Error uploading %s/%s" % (self.straight_url,
remote_filename))
BackendException: Error uploading
s3+http://sftpdaily0/duplicity-full.20120215T125153Z.vol1.difftar.gpg

BackendException: Error uploading
s3+http://sftpdaily0/duplicity-full.20120215T125153Z.vol1.difftar.gpg
Local and Remote metadata are synchronized, no sync needed.
Last full backup date: Wed Feb 15 13:51:53 2012
No old backup sets found, nothing deleted.