Re: Encoding for Robust Immutable Storage (ERIS) and Guile

[Ludovic Courtès]

Hi!

pukkamustard <pukkamustard@posteo.net> skribis:

> I'm happy to announce guile-eris 0.2.0. This is a Guile implementation
> of "Encoding for Robust Immutable Storage (ERIS)" [1].

Yay, congrats!

> ERIS defines how an arbirtary sequence of bytes can be encoded into a
> set
> of uniformly sized blocks and an identifier (read capability). The
> blocks are encrypted such that the original content can only be
> decoded
> given the read capability. ERIS is a scheme for content-addressing in
> the sense that the read capability is deterministically computed from
> the content itself.
>
> This is done by splitting content into blocks, encrypting them and
> collecting references to blocks in a higher-level node (i.e. building
> a
> Merkle Tree). See the specification [1] for a detailed description of
> the encoding.

AIUI, this is exclusively convergent encryption, which is probably the
right choice for most applications anyway.

Block size is fixed; did you consider content-defined block boundaries
and such?  Perhaps it doesn’t bring much though.

> Encodings like ERIS are common in protocols such as Bittorrent,
> GNUNet,
> IPFS, et. al. ERIS decouples the encoding from any particular protocol
> or application, allowing content to be referenced regardless of
> storage
> and transport layer. For example ERIS encoded content can be stored
> and
> transported over IPFS [2], but also over HTTP or via an USB stick.

The IPFS example is nice!  There are bindings to the IPFS HTTP interface
floating around for Guix; would be nice to converge on these bits.

> ERIS is still "experimental". This release is intended to initiate
> discussion and collect feedback from a wider circle. In particular I'd
> be interested in your thoughts on applications and the Guile API.

Do I get it right that the encoder currently keeps blocks in memory?

Do you have plans to provide an interface to the storage backend so one
can easily switch between in-memory, Datashards, IPFS, etc.?

Thanks for the great work!

Ludo’.