[Duplicity-talk] support xz compression

[Radim Tobolka]

Hi,
I'd like to add XZ (LZMA) compression to Duplicity. I'd like to discuss, 
how to go about it, so that the solution is aligned with your high level 
plan for the project.

It would be based on lzma module in Python 3 and it's backport in Python 2.

The feature could be be activated with new option, 
--compression-xz=<on|off|<preset>>
on - turn on, use default preset 6
off - turn off, give user chance to override, if specified multiple 
times (in scripts)
<preset> - 0-9[e], turn on with given preset with meaning and effect as 
documented in lzma package and xz(1) man page

Upon archiving, if active with encryption on:
- adjust archive files' extensions in file_naming to have .xz.gpg extension
- in gpg.GPGWriteFile, turn off default gpg compression, run data 
through lzma compressor before feeding it to gpg
if not encrypting:
- adjust archive files' extensions in file_naming to have .xz extension
- output to file obtained by means of lzma.open() in gpg.GzipWriteFile()

Upon restoring (the option need not be present, the feature could be 
autodetected):
- detect XZ compression in file_naming.parse, 
set_encryption_or_compression function and perhaps set new flag on the 
ParseResults object
- activate XZ decompressor in path.DupPath.filtered_open() based on 
above flag

There may be issues with accuracy of --volsize feature, because lzma 
uses larger buffers. Let's see during testing.

I'm successfully running PoC along these lines for few months now, 
albeit with piping to external xz process.

And of course, I'll add battery of tests and entry in manpage.

Any ideas, comments, feedback?

Best,
Radim