[Duplicity-talk] archive format ideas

[Travis H.]

I just had an idea.

If there were filesystem-specific programs that could dump files in
S-expr formats,
you could pick a single tag (e.g. data) that referred to the content
of the file.
Other flags would be left up to the filesystems.  Filtering the tags
that you wanted
after dumping is trivial, and if anyone has problems one could write a helper
application that allowed you to define filters or transformations
without actually
writing any imperative-style control flow stuff.  The mathematically inclined
could use a functional language to define their transformations in a very terse
mathematical manner (see Ocaml).

The restore program has a similar filesystem-specific helper that simply
creates the files on-disk with the appropriate attribute sets, if they
are present.
At worst, it would merely replicate the data tag as the contents of the file.

There could be a simple convention for experimental or implementation-specific
values, like the MIME types that begin with "x-".

The S-exprs could also be annotated with tags designed to make accessing
the files within the dump resemble random access in a file.  They could be
absolute file offsets, which might be tricky since you're inserting
into the file,
or they could all come up front or at the end, and they specify offsets relative
to the actual content tags.  Or one could build an index that sat alongside the
file.  In this way it would resemble databases, where one file stores
the records
and another is an index to it.  This has the added benefit that you can rebuild
an index file given only the content, and you can build multiple indices that
refer to the same content file.
--
"The obvious mathematical breakthrough would be the development of an
easy way to factor large prime numbers.'' [sic] -- Bill Gates  -><-
<URL:http://www.lightconsulting.com/~travis/>
GPG fingerprint: 9D3F 395A DAC5 5CCC 9066  151D 0A6B 4098 0C55 1484