Re: [GNUnet-developers] Idea for file storage in GNUnet

[Christian Grothoff]

On 12/08/2012 04:16 PM, Milan Bouchet-Valat wrote:
> A new feature that is currently being discussed for inclusion in the
> Linux kernel is volatile ranges. The idea is that applications can mark
> mmap()ed memory ranges and their associated on-disk storage as low
> priority, meaning they will be discarded when the system comes under
> pressure. This would mean GNUnet could have, in addition to the standard
> permanent datastore, an optional storage that would grow as much as the
> kernel allows it to. When space is needed, the memory pages and their
> files would be automatically dropped, and the GNUnet process would be
> notified.
> 
> The whole thing is not set in stone AFAIK, but that might be of
> interest. See:
> http://lwn.net/Articles/522135/

That would be of interest not so much for the datastore (large-scale, GB
to TB of data) but for the datacache (medium-size cache, a few MB to a
few GB) which is by definition temporary/volatile --- restarting your
peer clears the datacache already.  I recently added a 'heap'-based
datacache plugin implementation to SVN HEAD; that was mostly for systems
where IO is really expensive.  Still, with modern multi-GB RAM, moving
the datacache into this kind of storage would be interesting.

But, based on what I read, this is way preliminary, as it will likely
take years for this patch to be available in mainline distributions.
Still, worth watching ;-).

Happy hacking!

Christian