Re: [Help-gnunet] INDIRECTION_TABLE_SIZE and download speed

[James Blackwell]

> On Fri, 6 Sep 2002, James Blackwell wrote:
> 
> > If person A with a T1 donates 5,000 unique files to a p2p network of
> > which there is immediate interest in 50 files of 5 megabytes each,
> > then the p2p system is going to try and get all 50 at the same time.
> 
> Well, perhaps person A just should refrain from offering more
> than he can support...

Maybe I'm not explaining my point of view correctly. The way I see it, a
dialup user can effectively serve an infinite number of files provided they 
are not popular, oh let's say the whole of project guttenberg. But that same 
user can't serve 20 200k gifs of Britney Spears nekkid precisely because they
are popular? Doesn't that seem counterproductive to only be able to serve
in volume things that are of no value to anyone else? I fully admit that I
don't have a doctorate in comp sci, but it would seem to me that the whole
point of p2p networks was to enable everyone to get data others may value 
to people that want it.

The problem is that GNUNet is by design causing data to get stuck on the
originating hosts by allowing that new data being published to be
concurrantly requested, causing all of the files of value to get stuck in
the tiny little straw of a pipeline most people have. 

Let me try this another way. Take your average file copy utility and look
how it works when multiple files are being transferred. Does it attempt to
copy all of the files concurrantly? Of course not! File transfer utilities
transfer files from point A to point B primarily serially because to do 
otherwise would result in nothing useful getting from A to B (after all,
how useful is 1/2 of a binary or a tarball?). 

For the most part this is not true for p2p networks, though there is one
vital exception: Getting the original data off of the originating node.
That's because there is only *one* place that has this data set. We can't
get it *anywhere* else yet. It easily follows that we would want that data
to get out in chunks of the smallest usable size possible -- in this case,
the original files published.

Am I just completely off base to think it bonkers that I can't (and
wouldn't) be able to get data out via gnunet on a T1 that I could with
apache on an isdn line? After all... on apache, a user can easily limit
concurrant connections to ensure that at least a few people can get a copy
of the files in question and hopefully mirror it. In the case of gnunet,
they can't get out if they are popular.

>                         People expect a technical solution to 
> everything, when user education would be more in order.

Perhaps people do, but I don't. I thought about this pretty hard before I
talked about it, and I wouldn't have brought it up if I didn't think this
was a serious flaw in the logic of p2p systems. Isn't the point of p2p
systems to redistribute popular data rather than trap it on the
originating host? 

I think this particular problem does deserve a technical solution. Look at
the ubiquitious safety razor -- a technical solution for countless men and
women that had habit of cutting themselves shaving.

I would state that the intuitive use of a caching p2p network is to be
able to publish and distribute as much "valuable" information as possible,
and to expect people to do the exact opposite of what they would do if
they put up a web server is not reasonable.

I can't help but ask. Are you really intending to tell the 50,000 people
that will be using gnunet two years from now that they can publish at 
will if nobody is interested in the published information, but to cut 
back to one new file at a time if on a dialup and to stick only to ten 
new popular files at a time on a T1, even though they could accomplish 100 
times that with a web server?

Would it not make sense if you put some changes into gnunet to 
that only a few files are fed out at a time so that while the 
originating host is busy getting the second file out, gnunet is busy
helping distribute the first one, and so forth and so on? Wouldn't this
actually *help* with plausible deniability, since at times the originating
host would actually be lying about what it has? 

> 
> >     else {
> >         denyfileupload;
> >     }
> 
> Then what? People might query with a hash. Do you plan to send
> them some kind of "busy, come back later" -message? When would
> that later be? Or perhaps "no, this file is not available 
> at the moment. Get another."? But which one? If we tell,
> that'd be... telling. When would the node respond to 
> searches with locally inserted matches? Suppose it did, 
> when there was little load. Now the results would propagate
> and be available to people (supplied by other nodes) even
> when there was load on your node and you were unable to
> supply the file, having to denyfileupload.

Bearing in mind that you are the gnunet developer and I'm not, I would say
that the proper answer would be to do the same thing that is done when a
host is busy at other times.

> I don't say it can't be done. I don't say it shouldn't
> be done. Only that its complicated and it just might not be
> worth it. Inserting lots of junk in a way pollutes the 
> keyspace and makes it harder to find 'the good stuff'.
> Perhaps it would be better to insert a little excellent
> stuff and support that.

I have two answers to this, and neither of them are in agreement with you. 

Look at it this way. To some people, the GPL is just meaningless
licensese while to some it is just a license and yet others the driving
force behind the proliferation of free software. Pictures of nudes is
strictly pornography to some, an example of free speech to others, and a
study in human sexuality to yet more. I personally value Plato's writings
on socrates, but most would certainly consider that junk.  No. I don't feel 
qualified to determine what is valuable and what is crap. I'm sorry for
wandering on a tangital issue, but I do have to ask. Do you feel that you
could make the distinction between that which has value and that which is
"junk"?

I figure that is something gnunet tries to do. Ideally, the more valuable
something is, the more it will be requested. However, for the reasons I've
mentioned above, I suspect the opposite will actually occur.

I fully admit that I don't have a phd in comp sci or a degree in
algorithmic design (or whatever you would call it). I'm working on it, but
I'm not there yet. Then again, even though I do not know how to fix cars,
I can tell when the transmission in my car is going out and get a mechanic 
to replace it. I couldn't possibly hope to implement this with my current
level of education. If I could, I'd either submit a patch or fork the
project. This is the next approach; bringing the subject up on the mailing
list where the idea can be poked and prodded to see if it has any merit.

Sorry if I have wasted your time.
-- 
GnuPG fingerprint AAE4 8C76 58DA 5902 761D  247A 8A55 DA73 0635 7400
James Blackwell  --  Director http://www.linuxguru.net