[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [Duplicity-talk] webdavs and large sync volume failing
From: |
address@hidden |
Subject: |
Re: [Duplicity-talk] webdavs and large sync volume failing |
Date: |
Thu, 18 Feb 2016 21:51:59 +0100 |
Hi,
noone an idea what this problem could be or where I could continue trouble
shooting?
Thanks for help.
BR,
Wolle
> Am 06.02.2016 um 09:09 schrieb address@hidden:
>
> Hi,
>
> I have reproduced the error with verbosity 9 and sent the logfiles to
> ede/duply.net.
> As stated in that mail it looks like this is a memory issue
> (/var/log/messages shows that oom-killer kicks in)
>
> In parallel I have now upgraded my system to
> duplicity 0.7.06
> duply 1.11.1
>
> and re-run the backup with backend lftp+webdavs as proposed below. This works
> fine without errors
>
> but
>
> If I check the status of this backup via the same backend lftp+webdavs it
> shows no backup present.
> If I change back to webdavs backend for the status command I can see the 1
> successfull full backup I created with the lftp+webdavs backend.
> I tested the same with a small backup repository, here the backend
> lftp+webdavs works fine both for the backup and the status command
>
> A quick investigation I did shows that the status command for the big backup
> (roughly 14GB of photos) has an issue when retrieving the file info from the
> webdav server. There is no data returned - in the log below there should be a
> long file list after STDOUT: but this is empty.
>
> ——— snip from log ——
> CMD: lftp -c 'source
> '/home/backup/tmp/duplicity-uzDqze-tempdir/mkstemp-yXoZe9-1'; cd
> 'backup/photo/1971-2005/' || exit 0; ls'
> Reading results of 'lftp -c 'source
> '/home/backup/tmp/duplicity-uzDqze-tempdir/mkstemp-yXoZe9-1'; cd
> 'backup/photo/1971-2005/' || exit 0; ls''
> STDERR:
> ---- Resolving host address...
> ---- 1 address found: xxxxxxx
> ---- Connecting to sd2dav. xxxxxxx port 443
> ---- Sending request...
> ---> PROPFIND / HTTP/1.1
> ---> Host: sd2dav. xxxxxxx
> ---> User-Agent: lftp/4.6.0
> ---> Accept: */*
> ---> Depth: 0
> ---> Authorization: Basic xxxxxxx
> ---> Connection: keep-alive
> --->
> Certificate: xxxxxxx
> Trusted
> <--- HTTP/1.1 207 Multi-Status
> <--- Date: Sat, 06 Feb 2016 06:55:09 GMT
> <--- Server: Apache
> <--- ETag: "xxxxxxx ="
> <--- Content-Length: 1857
> <--- Content-Type: text/xml; charset="utf-8"
> <--- Vary: Accept-Encoding
> <--- Keep-Alive: timeout=3, max=100
> <--- Connection: Keep-Alive
> <---
> ---- Receiving body...
> ---- Hit EOF
> ---- Closing HTTP connection
> ---- Connecting to sd2dav. xxxxxxx port 443
> ---- Sending request...
> ---> PROPFIND /backup/photo/1971-2005/ HTTP/1.1
> ---> Host: sd2dav. xxxxxxx
> ---> User-Agent: lftp/4.6.0
> ---> Accept: */*
> ---> Depth: 0
> ---> Authorization: Basic xxxxxxx
> ---> Connection: keep-alive
> --->
> Certificate: xxxxxxx
> Trusted
> <--- HTTP/1.1 207 Multi-Status
> <--- Date: Sat, 06 Feb 2016 06:55:19 GMT
> <--- Server: Apache
> <--- ETag: "xxxxxxx ="
> <--- Content-Length: 1368
> <--- Content-Type: text/xml; charset="utf-8"
> <--- Vary: Accept-Encoding
> <--- Keep-Alive: timeout=3, max=100
> <--- Connection: Keep-Alive
> <---
> ---- Receiving body...
> ---- Hit EOF
> ---- Closing HTTP connection
> ---- Connecting to sd2dav. xxxxxxx port 443
> ---- Sending request...
> ---> GET /backup/photo/1971-2005/ HTTP/1.1
> ---> Host: sd2dav. xxxxxxx
> ---> User-Agent: lftp/4.6.0
> ---> Accept: */*
> ---> Authorization: Basic xxxxxxx
> ---> Connection: keep-alive
> --->
> Certificate: xxxxxxx
> Trusted
> <--- HTTP/1.1 200 OK
> <--- Date: Sat, 06 Feb 2016 06:55:20 GMT
> <--- Server: Apache
> <--- Last-Modified: Mon, 01 Feb 2016 19:44:00 GMT
> <--- ETag: "xxxxxxx ="
> <--- Accept-Ranges: bytes
> <--- Content-Length: 0
> <--- Content-Type: text/html; charset="UTF-8"
> <--- Vary: Accept-Encoding
> <--- Keep-Alive: timeout=3, max=100
> <--- Connection: Keep-Alive
> <---
> ---- Receiving body...
> ---- Received all
> ---- Closing HTTP connection
>
> STDOUT:
>
> Local and Remote metadata are synchronized, no sync needed.
>
> ——— snip end from log ——
>
>> Am 02.02.2016 um 21:27 schrieb address@hidden:
>>
>> On 02.02.2016 21:18, address@hidden wrote:
>>> Hi,
>>>
>>> I am trying to use Duplicity and duply to backup appr 60GB data to webdavs
>>> storage.
>>>
>>> The script is running on a raspberrypi with raspbian jessie
>>>
>>> Small backups are working.
>>> I have tried to run this backup in one go, it failed; I then split into
>>> 15GB chunks it failed again with exit code 137
>>>
>>> Any ideas what to do / where to look for?
>>
>> can you run duplicity in max. verbosity '-v9' and send me the _complete_
>> output privately?
>>
>>> Is the webdavs access not stable enough for such backup?
>>
>> should make no difference. the backup is split into volumes of the same size
>> anyway.
>>
>>> Is it maybe a performance issue with the raspberry
>>>
>>> btw I started the backup script via a cron job, it was then running several
>>> hours before it failed.
>>>
>>
>> what's your duplicity version? with the latest duplicity you can install
>> lftp and use that as alernative webdav backend via lftp+webdav://
>>
>> ..ede/duply.net
>>
>>
>
- [Duplicity-talk] webdavs and large sync volume failing, address@hidden, 2016/02/02
- Re: [Duplicity-talk] webdavs and large sync volume failing, edgar . soldin, 2016/02/02
- Re: [Duplicity-talk] webdavs and large sync volume failing, address@hidden, 2016/02/06
- Re: [Duplicity-talk] webdavs and large sync volume failing,
address@hidden <=
- Re: [Duplicity-talk] webdavs and large sync volume failing, edgar . soldin, 2016/02/19
- Re: [Duplicity-talk] webdavs and large sync volume failing, address@hidden, 2016/02/21
- Re: [Duplicity-talk] webdavs and large sync volume failing, address@hidden, 2016/02/22
- Re: [Duplicity-talk] webdavs and large sync volume failing, edgar . soldin, 2016/02/23
- Re: [Duplicity-talk] webdavs and large sync volume failing, address@hidden, 2016/02/23
- Re: [Duplicity-talk] webdavs and large sync volume failing, edgar . soldin, 2016/02/23
- Re: [Duplicity-talk] webdavs and large sync volume failing, address@hidden, 2016/02/24
- Message not available
- Re: [Duplicity-talk] webdavs and large sync volume failing, edgar . soldin, 2016/02/23