[Top][All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [Help-gnunet] INDIRECTION_TABLE_SIZE and download speed
From: |
Christian Grothoff |
Subject: |
Re: [Help-gnunet] INDIRECTION_TABLE_SIZE and download speed |
Date: |
Sat, 7 Sep 2002 15:54:30 -0500 |
User-agent: |
KMail/1.4.1 |
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
On Saturday 07 September 2002 07:55 am, Igor Wronsky wrote:
> On Fri, 6 Sep 2002, James Blackwell wrote:
> > The problem is that GNUNet is by design causing data to get stuck on the
> > originating hosts by allowing that new data being published to be
> > concurrantly requested, causing all of the files of value to get stuck in
> > the tiny little straw of a pipeline most people have.
>
> I see your point. I just don't see a neat technical solution.
> A straightforward way would be for the node the keep track of
> which blocks are indexed locally and which blocks belong to
> which file. Once some block is accessed, prevent the downloading
> of blocks belonging to other locally indexed files for a certain
> period of time. The extension to multiple-files-simultaneously
> is straightforward.
Bad idea. When would you decide that content is now popular enough to just
serve it at all times? Also, by not answering the request, all that happens
is that you get it later again (after another 50000 nodes have been queried),
so you just wasted bandwidth that you tried to conserve. If you get a request
and you can answer (bandwidth limits, etc), do answer. Only then the other
nodes get the chance to replicate the content. Locking away certain files
(even for a limited time) is always just a bad idea, it's wasting storage
space.
> > For the most part this is not true for p2p networks, though there is one
> > vital exception: Getting the original data off of the originating node.
> > That's because there is only *one* place that has this data set.
>
> On freenet that doesn't hold. Inserted content is distributed
> to TTL hosts on insert. I think gnunet should push content out
> as well, but I haven't been able to convince CG of how that'd
> fit the economy model - because I don't know a good answer
> myself. Content pushing would solve the issue partly though,
> because the load would be instantly distributed, and we
> are inserting just one file at a time.
What if somebody inserts content while being off-line? What I am really not
convinced of, is that the scenario 'new popular content inserted, everybody
wants it, node can not serve' is the real problem (or anywhere near it).
Because the popular content would be send to the first requester and
*instantly* replicated on the way (similar to freenet). So this is definitely
not the problem. The problem is, that if the person initially providing the
content goes off-line (say forever) after half of the file has been
downloaded. If the content is also popular, many people will try to download
the file, at first even get decent speeds, but after 1/2 of the file has been
retrieved (which is useless due to the random order of the blocks), the
download speed drops to 0 because the rest is *not available*. At all, ever.
And there is no way to prevent this from happening since you can not force
the originator to go back on-line.
The only solution is to only "allow" a download after n replicas of the entire
file exist in the network and we can guarantee with reasonable probability
that not all of them can disappear. But this is still not a good solution
since malicious parties could still insert half a file and claim that it's
the whole thing. Anonymity constraints prevent us from tracking such
malicious parties. Signing individual content with 'I guarantee that it'll be
there' does not help either. *Directories* which collections of good content
will do a better job (the person creating the directory 'vouches' for content
availability, if you got 3 files from the directory and they were all ok, the
other 300 will probably also be ok). Note that we would not even need
pseudonyms or anything else for these directories (the current file-encoding
will suffice for integrity).
> For this particular problem, I don't know if CG wants
> to add NO_GO_NOW -replies to the protocol, but it'd seem
> to require that, or otherwise the users won't know
> what gives and think the net even worse as now (when
> they can get that 200cps trickle).
Definitely not. What we rather need is some cleverness in the download-code to
tell the user that 'the content is not available with probability X%'. And
there are some improvements that I want to do to the routing code to find
available content faster (fewer hops), but routing issues seem to be beyond
the point of this discussion.
Christian
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.6 (GNU/Linux)
Comment: For info see http://www.gnupg.org
iD8DBQE9emeH9tNtMeXQLkIRAoxdAJ4t9oUOj/kMgZnaOidbUuWvvu/f+wCghvC8
gWdSylb74dJ5Jp0/myd8SP8=
=Lj6l
-----END PGP SIGNATURE-----
- Re: [Help-gnunet] INDIRECTION_TABLE_SIZE and download speed, (continued)
- Re: [Help-gnunet] INDIRECTION_TABLE_SIZE and download speed, James Blackwell, 2002/09/06
- Re: [Help-gnunet] INDIRECTION_TABLE_SIZE and download speed, Igor Wronsky, 2002/09/06
- Re: [Help-gnunet] INDIRECTION_TABLE_SIZE and download speed, Christian Grothoff, 2002/09/06
- Re: [Help-gnunet] INDIRECTION_TABLE_SIZE and download speed, Tracy R Reed, 2002/09/07
- Re: [Help-gnunet] INDIRECTION_TABLE_SIZE and download speed, Christian Grothoff, 2002/09/07
- Re: [Help-gnunet] INDIRECTION_TABLE_SIZE and download speed, Christian Drechsler, 2002/09/09
- Re: [Help-gnunet] INDIRECTION_TABLE_SIZE and download speed, Christian Grothoff, 2002/09/09
- Re: [Help-gnunet] INDIRECTION_TABLE_SIZE and download speed, Igor Wronsky, 2002/09/06
- Re: [Help-gnunet] INDIRECTION_TABLE_SIZE and download speed, James Blackwell, 2002/09/07
- Re: [Help-gnunet] INDIRECTION_TABLE_SIZE and download speed, Igor Wronsky, 2002/09/07
- Re: [Help-gnunet] INDIRECTION_TABLE_SIZE and download speed,
Christian Grothoff <=
- Re: [Help-gnunet] INDIRECTION_TABLE_SIZE and download speed, Ingo Ruhnke, 2002/09/07
- Re: [Help-gnunet] INDIRECTION_TABLE_SIZE and download speed, Christian Grothoff, 2002/09/07
- Re: [Help-gnunet] INDIRECTION_TABLE_SIZE and download speed, Tracy R Reed, 2002/09/08
- [Help-gnunet] Expiring search results?, Igor Wronsky, 2002/09/08
- Re: [Help-gnunet] INDIRECTION_TABLE_SIZE and download speed, Tracy R Reed, 2002/09/08
- Re: [Help-gnunet] INDIRECTION_TABLE_SIZE and download speed, Ingo Ruhnke, 2002/09/08
- Re: [Help-gnunet] INDIRECTION_TABLE_SIZE and download speed, Christian Grothoff, 2002/09/08
- Re: [Help-gnunet] INDIRECTION_TABLE_SIZE and download speed, Tracy R Reed, 2002/09/08
- Re: [Help-gnunet] INDIRECTION_TABLE_SIZE and download speed, Christian Grothoff, 2002/09/08
- Re: [Help-gnunet] INDIRECTION_TABLE_SIZE and download speed, Tracy R Reed, 2002/09/08