Re: [GNUnet-developers] vsftpd/wget/gnunet

[Igor Wronsky]

On Sun, 29 Dec 2002, Jan Marco Alkema wrote:

> In the past I have programmed some “flat file database” systems. At that
> time I was very happy with it, but now (in helicopter view) I see that I
> programmed things with are better worked out in real database programs.
> I have tested 10 million of file footprints (size, name, MD5, path) in
> MySQL. It was very fast, if you put indexes on the attributes.

Well, here we are, still, at the ground, looking at you flying
your nice helicopter. ;) But more seriously, I'm sure the gnunet
team will consider some more involved database manager the moment
it becomes necessary, that is, when it takes such a turn where gdbm
or tdb won't do anymore. Meanwhile its all down simplicity and efficiency.
If you can, in terms of algorithms and their complexity - or some
published, serious empirical results - make a convincing argument
on the lines of "indexing algorithm A in dbmgr X is better than
algorithm B in gdbm because..." and references, that would probably
be a good start to turn our heads. In addition you should perhaps
study gnunet design and the way we block and use data in AFS, I'm
not certain if you understand that, and why the current database
mechanism is sufficient for us. If, in your opinion, the whole our
current way of handling files is bad, you should state that
clearly and present alternatives and convincing argument for
them.

Till then, there is no point continuing this discussion on
just feeling-based grounds.

> If you have for example 15 million host keys and 20000 million footprints of
> files than you get a lot of problem if you haven’t a good database design
> with good indexes.

Well, we shall see whether there will ever be even 1000 gnunet
host keys. :) Besides, I've a feeling the current gnunet routing
can't manage nearly the scale you suggest, so that would have
to be solved first. :(

> I think a good suggestion. But I think that if you know how to implement
> jdbc/odbc it is not so difficult as it may seem.

Its not difficult, I admit that. Its just that if there's no
point to it, it shouldn't be done.

> I don’t see the advances of the bloom filter yet?

Its quite transparent. Perhaps we should collect some statistics
about its operation.

> If someone will help to make a superiors file sharing system let me know.
> Maybe you don’t get it: I want to make a mix between the Napster (central
> database) idea and Gnutella (distributed database)idea. Everyone can have a
> central database (by synchronization of the databases of the other nodes).

You might be interested in a project called "mldonkey" which was
recently brought to my attention. They try - at the moment -
to support different p2p-fileshare systems in a single framework,
the systems including e.g. edonkey, overnet and some others. They
might be more open to those ideas. While I think that taking
over the world is a fine goal for anyone, there's enough problems
to solve in gnunet in realizing the anonymous information sharing
in scalable and robust manner. A primary goal for gnunet might be
to be competitive or better than freenet, and if it were 'not much
worse' in efficiency than edonkey or overnet, that would be a nice
bonus. After that goal is realized it might be considered,
again in my humble opinion only, what else should/could be done.

But in addition its questionable if a 'hacker effort' could
pull together a world wide unified file storage (there's certainly
papers about such ideas) or whether it will require a standardizing
committee and strong industry participation.


Igor