Re: [GNUnet-developers] Replacing lookup

[Igor Wronsky]

Thanks for your thoughts, everyone. Here's some answers.

1) Jason, the typical way of using MySQL is linking to the client library.
The user still needs to have the MySQL server running and maintaining the
databases. If there is a solution not requiring mysqld, I don't
know about it. Somebody could find out.

2) Chris, your summarization is accurate. If the freenet project had
problems with the datastore, I don't see we could build it nice and
clean from the scratch without significant time investments as well.
Its practically a filesystem, although special purpose one. Automatization
of MySQL table/user creation is possible. Your assumption that I
was about to leave bloomfilters "outside" was correct. Iterator
is no problem.

3) Tom, reason for not mentioning libdb* was that I didn't know
of it. By a quick glance it doesn't seem to be supporting multiple
keys for a single database (thats what we need: indexing by data
hash and indexing by priority) so its similar to gdbm in that
fashion. I don't also know whether it keeps data and structure
separate. The first issue would mean that we'd have to do a kludge
to get two indexes and thus a tool for maintaining the consistency
of those (e.g. a separate database for key=priority,data=hashcode
pairs). But what I'm wanting is just an interface change and
restructuring of the code, if somebody wants to pull libdb
module together, fine by me. And as Chris said, its nontrivial
to borrow code from project to another in this case, especially
supporting multiple indexes in one-index system might require
large overhaul of tdb or gdbm.

4) Tracy, we already practically depend on tdb or gdbm. The
directory storage method is very inefficient for gnunet, atleast
unless the user runs an underlying filesystem with blocksize 1k.
And still... The best option would be if someone would come up
and code a rock-solid and portable db for just what we need,
but I don't see any volunteers. ;)

5) Krista, we would be requiring MySQL (or some other heavy
solution) in practice unless someone fixes the lookup+gdbm/tdb
combination code. Wasn't that the case that when it fills
up, thats that. ;)

6) Niklas, MySQL ram or cpu usage hasn't been a problem for me,
and the startup is "instanteous" with 800mhz machine. With 450 megs
of memory, its typically sits somewhere around 2%. So its
imho not tragic.

Ok, not much more now. I'm not going to jump right away in
to the refactoring, as I'm not exactly sure how to do it cleanly;
after all, we still want the old tdb/gdb/etc code and
gnunet-check to be usable.


Igor