Re: [Bug-gnupedia] From the "Department of Wacky Ideas"

[Bob Dodd]

> Um, I think this is a very wacky idea, and a great
> model being applied to the wrong situation.

It is indeed a wacky idea :-)) but I'm still not convinced
it'completely wrong...

> You see
> with something like SETI, or Gnutella, it doesn't
> matter if one server goes off line. In fact, so long
> as some people are there, it still runs. 
> But thats
> nothing like Alexandria or Nupedia. Imagine if you
> found the server(s) holding the nugget of information
> you wanted had gone offline!

That was point (1) in my list of disadvantages. The solution proposed
was to duplicate the data across multiple servers (which I fully accept
has a list of disadvantages too).

Ignoring for the moment the total concept, we still need something
similar... that's traditionally what mirrors give you (but that does
require your users to "sort themselves out" and choose an appropriate
mirror. I'm not sure how "modern" a solution this is though: why should
the general public have to go hunting for working mirrors?). The
alternative to a mirror in high availability systems (when you have 3
or more servers available) is to distribute your database between the
machines. Depending on your availability requirements, this also means
that each machine reuqires less storage capacity, and with careful
partitioning of information, less processor and memory resources too.
Of course how much you need this sort of partitioning depends on what
response time you require, and how much load you want on each machine.
It is a neat way to reduce hardware requirements though...

It would certainly improve the reliability, even within a single
location, and people wouldn't need to go looking for mirrors so often.

> 
> Then there's the databases. It would be an incredible
> task to first of all have one central article
> submission server, that would then somehow transfer
> the article + media to the correct server, and then
> reference that somewhere (where?). You would have to
> have the web site on one server, but ho would you
> search the database? It would take forever to to a
> cross-web search, as Gnutella aptly shows. Even
> Napster was a bit slow at searching, and it had the
> info on a central server! You said yourself you'd have
> problems keeping check on the baby servers too.

The location of content doesn't affect database searches: the location
of the indexes does. I would hope that the size of any index (OK, we
may have a lot of imdexes :-) will be signifcantly smaller the the
content indexed. Therefore, most servers should be capable of
carrying/caching a large part of the index. So similar subsequent
subject area searches should speed up.

> No, you'd get a very, very messy setup with redundant
> info on some servers, half the information missing,
> and a very confusing setup to put up, expand, use, and
> program.

Difficult to program: certainly more difficult than the alternative.
And more difficult to maintain. I think I made that point too.
>From a user's perspective, he/she shouldn't notice the difference, that
would be our problem.

As for redundant info, I think that's the point :-))

> I see no disadvantages in a central server with
> several (as many as possible) mirrors. 

In a word: performance.  You're going to generate a lot of traffic to a
few locations, and you'll need some beefy servers to support searches
on the database. Large coporations spend a lot of money on
high-performance machines to handle their database traffic, I don't see
how we would be any different. After all, we don't just respond with
the index information a la google, we supply content too.

That said, of course it can be done with less h/w than commercial
outfits, we don't have their performance criteria. Public projects like
ours tend to produce better, tighter code too... But it's still going
to need a lot more hardware & money than the "wacky" approach. On the
main server and at each mirror too.

I'm also sure we can get the sponsorship we need, but it will also
leave us dependent on those sponsors too.


> The central
> server would be, I assume, in the US, and even their
> control-freak government can't shut down something
> like this.

Actually gthe way things are going, the US may soon be the only place
they can't shut things down. 

> Then there are plenty of countries in the
> world where it could be mirrored. And I don't see how
> the baby-server idea gets around the "Chinese
> firewall" which prevents browsing of the web properly
> anyway!

It just provides more opportunities to find "workarounds".
The people who devised the firewall aren't rocket scientists, and the
more portals you can create, the more chance of finding loopholes.

> All you need is some sponsoring, and a really decent
> setup with unlimited capabilities could be set up
> easily with Perl, mod_backhand and mySQL. It would
> handle any amoutn of data thrown at it, provided we
> had the HDD space, and mirrors would all be set up in
> the same way.

True. It's well understood. It's relatively easy to set up. It just
takes money to setup and maintain.


> This way the resource is fast, reliable, and problem
> free. Why change it with a model that would be fraught
> with possible problems?

I wouldn't, unless it worked, that's why I said that you start from the
conventional server/mirror model. But I think we'd learn a lot from
configuring one of our servers like this, and after that experience,
looking again at the idea. 

Anyway Tom, it's nice to get such quick answers back, even if it's only
to say you hated it :-)) Thanks.  

/Bob Dodd




__________________________________________________
Do You Yahoo!?
Yahoo! Auctions - Buy the things you want at great prices. 
http://auctions.yahoo.com/