Re: [Duplicity-talk] Use alternative diff backend

[Aaron Whitehouse]

Sorry that I missed this at the time.

On Jul 29 2018, at 4:19 pm, edgar.soldin--- via Duplicity-talk <address@hidden> wrote:

On 7/29/2018 16:48, darkdragon wrote:

Next generation file systems like ZFS and BtrFs allow storing snapshots with no additional costs. Calculating diffs is also extremely cheap.

can you give a command line or native python example to retrieve a diff by file?

Currently only a native format is supported (see btrfs-send:

https://btrfs.wiki.kernel.org/index.php/Manpage/btrfs-send). There are

some projects for decoding this stream and outputting it in various

formats (see https://github.com/sysnux/btrfs-snapshots-diff including

issues).

I think that a nice intermediate ground here would be to leverage these COW filesystem capabilities to avoid a full filesystem scan.

I have been toying with the idea of adding a way to specify the changed files/files for duplicity to scan, rather than duplicity scanning all files looking for changes.

My initial use case was an inotify-based implementation that always kept everything backed up, but it could similarly be used for this COW filesystem support if you could pull the list of changed files from the filesystem without a scan.

That has the advantage that it would essentially just need some tweaks to the filesystem walk, without changing any of the diff logic etc. It would also mean that you could run occasional 'normal' scans to wash-up anything that wasn't caught for whatever reason.

Kind regards,

Aaron