[GNUnet-developers] Designing a gnunet directory app

[Igor Wronsky]

On Wed, 12 Jun 2002, Christian Grothoff wrote:

> Well, I've found strange music and a jpg searching for 'mp3' recently, so 
> there is definitely some content. 

Apparently the industry doesn't want to get caught with its pants
down this time, but be right on the spot where everythings happening. ;) 

> You could also just create 'directories'. Insert a couple of files and post 
> the results of 'gnunet-search' (hash, length, crc, description) in a 
> 'directory' file. I would suggest to use 'directory' as the keyword for these 
> index-files for now. This way, we can use GNUnet itself instead of a webpage 
> as the indexing service.
> And I'm sure, there'll be some code to create directories automatically at 
> some point. Any volunteers? :-)

That would be within my capabilities and I'd be interested to
code one. However, I suggest it to be done untypically 
this time: mainly, design first. Here's some things that came
to mind.

There are several problems. First, a more general issue, how
can we get rid of obsolete directories? Suppose I insert one
with a keyword "directory" and include a timestamp. Could people
querying for "directory" get that same hit even a year afterwards, 
as hitting it will keep it at least in some sense popular? 
Does gnunet have mechanism to get rid of files that are 
often 'hit' but rarely if ever downloaded?

The next issue is, how should the app automate the creation
of such list? Should just a short script be hacked together
to direct the output of gnunet-search (on one or more runs)
into a textfile and insert that? Or should a variant of
gnunet-search be created thar would collect found keys to a 
file, and insert a new directory after a certain amount of 
new keys have been found? Is it benefical to store the actual 
keys, which can quickly disappear from the network? And what 
are the gains/losses against a solution where only some good 
keywords would be listed, perhaps with statistics of the time of 
list creation (e.g. keyword: test hits: 138 desc: miscellaneous 
junk for testing purposes)? That would just give people hints
of what to look for, but different people rerunning those queries 
of popular words might be costly. In the end, of course, when
gnunet encompasses the whole planet (and the colonies), a
keywordlist will be unnecessary as *any key* will find plenty. ;D

Should compression be used on the directories?

Naturally the directory listings should also be machine (eg gui) 
readable. The app should be able to retrieve newest up-to-date lists.
Perhaps lists could be signed by sender, using a handle perhaps. 
Of course trivially done that would not be proof against attacks,
but if most people are benevolent, it'd enable 'fan communities' 
to follow directories created by some famous persons. ;)

Anyway, the solution starts to remind me of frost on freenet, 
performing some of the issues mentioned above, essentially being a
directory creation, maintaining and retrieval tool, all in
one, packed with messaging capabilities. Frost would be very easy
to port to gnunet but it seems atleast to me that a straight
or trivial port shouldn't be attempted before the issue
is well thought about. Such "polling application" hasn't done 
much good to freenet, and I can't downright say what it 
would do to gnunet -- I don't know the guts of either system
too well. The thing is though that if some similar app 
isn't done intelligently for gnunet, it will be eventually made 
brainlessly by a third party and bad things will happen (on freenet 
there's now atleast four ways to transmit and find files: private, 
not announced anywhere, "freesites", "frost" and "fmb". this
has unnecessarily split the available resources between
incompatible methods of announcement/retrieval/etc) :(

Everyone, what do you think about these things? Any additional 
ideas? What should the dir app go for? Something like frost w/
channels/interest groups, or just a small script to pack
out found matches as ad hoc directory files?

Plenty of questions. ;D


Igor