[GNUnet-developers] an interesting bug with respect to performance...

[Christian Grothoff]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi all!

I've just fixed an interesting bug that had a significant impact on 
performance, and I thought I should share...

The code did the following:
1) client sends query with TTL x (say 5s)
2) gnunetd puts query in routing table, performs local lookup,
    (and remote lookup, but that is beyond the scope of this 
    problem).
3) local lookup succeeds instantly, reply is send to client
4) client sends next request (max requests concurrently pending
     is 256 or so)
5) after several iterations  1-4 in a few ms, the routing table is full
     since the slots are blocked for TTL seconds, thus gnunetd
     drops (!) the request from the client (can not route, routing table
      full)
6) the client tries again with a higher TTL
7) when the slot is finally free (previous ttl expired), gnunetd 
    puts the new request in the slot, the lookup succeeds instantly.
    But the slot is now blocked for even longer (higher TTL)

Over time, the average TTL used by the client increases, the routing
table is always full, the download speed goes to hell.

FIX: if the routing table checks if a slot should be discarded, take
the number of replies send so far into account. If we have already
send a reply, replace regardless of ttl (ite->seenIndex > 0). Since this 
behavior is better for the (more common) downloads (as opposed to the
queries which can have multiple valid results), this fix should be valid.

Note that it would be better in this case if GNUnet could distinguish between
queries that search and queries that download. While this distinction could
achieved and would have other benefits, it would also have some security 
implications. I'll try to get some more feedback on the trade-offs involved to 
make up my mind :-)

Summary:
The result of the fix is that a 3 MB file can now be downloaded from the 
localhost in less than 14s (prior reports were closer to 14 minutes...).

Christian
[Note that the code to do this is currently not available for 0.4.6, CVS only]
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.6 (GNU/Linux)
Comment: For info see http://www.gnupg.org

iD8DBQE9fCUu9tNtMeXQLkIRAjk4AJ915Mpcw5P1Jf1ekwmWnP22xMun6wCePbOd
pewCvHVoHv2GHrFuNB6/Pjs=
=50FO
-----END PGP SIGNATURE-----