Re: [rdiff-backup-users] initial rdiff-backup for large repository over Internet (and connection lost)

[Dominic Raferd]

On 25 February 2016 at 01:19, Adrian Klaver <address@hidden> wrote:

On 02/24/2016 01:57 PM, Nicolas wrote:

Hi all,

I'm an happy rdiff-backup user for many years.

Years ago I setup a rdiff-backup from a WWW server to a local server.
Internet connection is limited to 2Mbps between the 2 servers.
As /var/www was initially empty, rdiff-backup as done its backup job
everyday without problem, with many sites added in /var/www.

Now /var/www is 14G large and I need to fresh start the backup
(destination directory has been deleted).

Bad thing is that trying to first sync source and destination via the
command :
nice -n 19 rdiff-backup --force $SOURCE $DEST::$DESTDIR
always ends with network (Internet error) like these :
Found interrupted initial backup. Removing...
Write failed: Broken pipe
Fatal Error: Lost connection to the remote system

What would be proper way to first sync rdiff-backup ?
Would a previous rsync help, so that only meta-datas would be written by
rdiff ?

I do not use rdiff much any more, as I switched over to borg-backup(http://borgbackup.readthedocs.org/en/stable/), so I am a little rusty on my advice. In addition to my previous post, I now remember something else I used to do. That is rdiff from the source to a local destination, then rsync the local destination to the remote destination, finally switch the backup to use the remote instead of the local destination. rsync is more forgiving of network interruptions.

I think that is good advice, it is the strategy I follow with my TimeDicer - a local 'primary' rdiff-backup server and a remote 'mirror' server that is synchronized by a script on the primary which uses rsync.

Alternatively it is possible to tweak the ssh settings to help rdiff-backup maintain a session over the internet, still I don't think it is such a robust solution.