gnunet-developers
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [GNUnet-developers] more efficient sharing of similar data


From: Christian Grothoff
Subject: Re: [GNUnet-developers] more efficient sharing of similar data
Date: Sat, 17 Aug 2002 15:33:17 -0500
User-agent: KMail/1.4.1

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Saturday 17 August 2002 06:25 am, Martin Uecker wrote:
> In the preprocessing case each place where the rolling checksum
> hits the magic value would be moved to a 1k boundary. This scheme
> would work without modifying one line of code in gnunet.

I kind of doubt that - if we take on-demand encoding into consideration (where 
GNUnet receives a request and then looks at 1k of the (unmodified) file in 
plaintext on the drive and sends back the reply).

> Integrating it into the client has the advantage that the user
> can serve unmodified files (this is an argument for putting ECC
> into the client to). But the index where the hashes are stored
> must be extended with information where each block starts.
> (10 bits of additional information for each 1k block)

You can have 10 bits, and only 10 bits:       :-) 

/**
 * Type of the content index file entries.
 **/
typedef struct {
  /** the double-hash of this entry */
  HashCode160 doubleHash __attribute__ ((packed));
  /** the current rating of this content (network byte order)*/
  unsigned int importance __attribute__ ((packed));
  /** the filename, 0  for entry not used, -1 for data/content shared, > 0 for 
on-demand encoding (network byte order)*/
  unsigned int fileNameIndex __attribute__ ((packed));
  /** the offset in the file, offset for on-demand-encoding (network byte 
order) */
  unsigned int fileOffset __attribute__ ((packed));
} ContentIndex __attribute__ ((packed));

The last 10 bits in 'fileOffset' are currently always 0 (1k alignment!). 

> > | Also I'm not sure how tightly you can fit all blocks into the 1k
> > | scheme (which would of course be best since you gain anonymity if all
> > | blocks are truely uniform in size).
>
> Yes, this is critical for efficiency too.
>
> I have to think about the idea some more but I will provide
> patches to play with next month.

Well, I hope you'll give us a pile of documentation / descriptions with them, 
because this sounds like it's going to be *very* complicated (but promising, 
nevertheless, could solve one of the problems that we had with our 
tree-encoding which was that inserting a single character at the beginning 
would make the entire thing look totally different). So I'm looking forward 
to this.

Christian
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.6 (GNU/Linux)
Comment: For info see http://www.gnupg.org

iD8DBQE9XrMN9tNtMeXQLkIRAokiAJ4gSH1Wl3KY/ja8n95eG4rs02kOOACcCAyd
6l04XQsvXrsNuNvt5AyH7l4=
=OWIJ
-----END PGP SIGNATURE-----





reply via email to

[Prev in Thread] Current Thread [Next in Thread]