Re: [Help-gnunet] I'm not sure what going on

[Christian Grothoff]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Saturday 17 August 2002 02:11 pm, address@hidden wrote:
> Hi gnunet people,
>
> since yesterday I'm running gnunet 0.4.5 / libextractor 0.1.1.
> It's not the first time I'm running gnunet. I run gnunet from time to
> time for some days. In most instances when a new version is made,
> to see the progress to the way to the long road to a useful P2P program.
>
> Search is much faster now, but download ist still extrem slow.
> The download speed is still far away from practical.

Are you sure you downloaded a file that was actually available (host online) 
at that time? Which speed did you get? What did you expect?

> In what development stage will you see gnunet now ?
> Is the protocol settled, and further work will 'only' made to the
> implementation ?

The protocol is not settled, and the implementation of the current
protocol is also not the final answer. 

> I can't find any statement about this in FAQ or Documentation.

I'd rather document what we have than what we may have in the future; software 
always changes, no need to say that :-).

> gnunetd logged (in level 5) in 11,5 hours that messages:
>
> 111 times
> CONNECTION: decrypting message from host XXX failed, no sessionkey!
> 111 times
> Can not decrypt packet from host XXX: no session key? (-1).
> 6 times:
> CRC error. Dropping packet from 37B4XXX-AAA. AA. AA. AA: 2086.
> 97 times:
> CRC error. Dropping packet from 7A5AXXX-BBB.BBB.BBB.BBB: 2086.
> 3 times:
> CRC error. Dropping packet from 81E1XXX-CC.CCC.CCC. CC: 2086.
> 3566 times:
> CRC error. Dropping packet from 8D5CXXX-DDD.DDD.DDD.DDD: 2086.
> 40 times:
> CRC error. Dropping packet from 9782XXX-EE.EEE. EE. EE: 2086.
> 2 times:
> CRC error. Dropping packet from BB4BXXX-FFF. FF.FFF. FF: 2086.
> 74 times:
> CRC error. Dropping packet from C40EXXX-GG.GGG. GG. GG: 2086.
> 21 times:
> CRC error. Dropping packet from CB7BXXX-HHH. HH.HHH.HHH: 2086.
> 3 times:
> CRC error. Dropping packet from D1F1XXX-III. II. II. II: 2086.
> 37 times: SKEY: Session key exchange denied, slot busy.
> 10 times: WARNING: session key failed verification, ignored!
> 97 times:
> No connection information for host available
> (/var/lib/GNUnet/data/hosts/3610XXX
> 97 times:
> Could not get connection information for host 3610XXX

This amount of problems sounds normal. In particular if a node is new and 
joins the network, there are some 'rough edges' in the way the protocol is 
*implemented* that will cause "miscommunications" like the above (like a node 
sends your node a key and your node says "I don't know you (yet)"). For some
of these issues, we may be able to tweak the implementation to avoid them, 
but again, they are not a problem (you received 240.000 messages, and only
5.000 (2%) were 'misunderstood'). Yes, 2% is not great, but if you're online 
longer, the percentage would drop.

> Maybe my stats are in interest too:
>
> Node-to-Node (udp):
> #HELO      received:              3814  send:              5002
> #SKEY      received:               299  send:               472
> #PING      received:                48  send:              1507
> #PONG      received:                49  send:                48
> #3QUERY    received:           1545704  send:           1971585
> #CONTENT   received:              6124  send:              7933
> #TIMESTAMP received:                 0  send:                 0
> #SEQUENCE  received:            240307  send:            246691
> #NOISE     received:            248616k send:            233614k
> #HANGUP    received:                 4  send:                11
> #octets    received:            351742k send:            355376k
>
> Server-Client (tcp):
> #QUERY     received:             28619  send:                 0
> #CONTENT   received:                 0  send:               724
> #INSERT    received:                 0  send:                 0
> #INDEX     received:                 0  send:                 0
> #LISTFILE  received:                 0  send:                 0
> #FILEINDEX received:                 0  send:                 0
> #STATREQ.  received:              3509  send:                 0
> #STATREPLY received:                 0  send:              3508
>
> Server Statistics:
> Shared files       :                14
> Size of shared data:              3097k
> Connected hosts    :                13
> Uptime             :             43198s
>
>
> Anything I must worry ?

Not really. Look closely at the numbers. We have over 12h an average of
35 queries *per second*, and this while suffering from the (well-known) 
problem that the current code is very inefficient because it generates too
much noise: over 70% of your network traffic is noise! So if we could get the 
noise down to "0%" (over-idealistic, but a nice experiment for thoughs), 
GNUnet could process 100 queries per second (at least bandwidth wise, which 
is typically the limitation) on your node. And there are some ways to improve
upon the current state of the protocol if we're willing to waste some more CPU
cycles. 

About 100 qps, let me quote an article from saloon.com (09/29/2000):
"Earlier this month, researchers from Clip2 Distributed Search Services 
published a report documenting another major flaw. According to the report, 
the Gnutella network is only as strong as the bandwidth of its weakest users 
- -- too many people using 56K modems are being required to channel traffic, 
causing the entire system to slow to a crawl. The report pegs the 
"scalability barrier" at 10 queries per second; any more, and the network 
will break down. "

A query in GNUnet is at least 28 bytes long (not in the current protocol, 
there we have 52 bytes, but theoretically), that would roughly mean that we 
could pipe 256 queries per second (56 * 1024 / 8 / 28) through a modem line. 
Yes, there is some additional overhead, but this is the order that we could 
reach. The problem here really will be to process 256 queries/second locally. 

To summarize: no, we're neither at the end of protocol nor application 
development, but the current protocol and the current application can already 
perform what we set out to do - anonymous filesharing. It may not be as fast 
as you (or we) would like it to be, but there is room for improvements. One 
particular point would for example be a more aggressive content replication - 
which would speed up downloads and increase reliability. Just keep in mind 
that a 'secure' p2p network will probably never be nearly as fast as a plain 
TCP download from your typical webserver.

Christian
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.6 (GNU/Linux)
Comment: For info see http://www.gnupg.org

iD8DBQE9XqxI9tNtMeXQLkIRAkOoAJ4gfXzlPUWxGw/0DsrK0Rs/1EzZ7gCfX8Nw
AI4HQ4KaIzMaAWpKMruWnrE=
=OUqD
-----END PGP SIGNATURE-----