Re: [Bug-gnupedia] Architecture Questions

[Bob Dodd]

[snip]

> but in C),  if the db goes down the system becomes
> unsearchable,  which makes it effectively useless,
> unless the user wants to remember the uid# of each
> file you need.
> This was something that came up when me an Thom were
> figuring out our  ``proposal'', but we decided against
> it for just this reason.
 
First, I wouldn't expect a user to even *know* there is a uid for each
file: the internal storage format should never be visible to users.

Second, if the db goes down, the main thing is to have a mechanism the
gets it back up quickly. We can also limit the possibility of the db
going off-line by designing our system as a high-availability platform,
with mirroring of the db within each server. That can either be
mindless full mirroring of all data, or by sharing the content between
servers as I've suggested before. Since we're now talking about
reliability, I'll repeat the model:

"If you want an example of a real commercial company that uses
this apporach for high availablity, look at people like ObjectStore who
use this divide-and-conquer approach: if you had 3 servers, they would
divide your data into 3 parts, and each machine would hold 2/3 of the
whole so that machine A would hold parts a & b, machine B would hold
parts a & c, and machine C would hold parts b & c (or some combination
of the same…) So if any one server fails it is still possible to
continue.   With more machines, they would make smaller parts, and each
machine would perhaps hold more parts."

What I didn't say before, is that you usually have some form of router
that also load balances incoming database requests between the machines
(and if you intelligently partition your data, you can limit the
interactions beween the machines).

With that approach, you can take down individual machines for a short
time e.g. for maintenance/upgrades without comprimising the server
integrity, but without having to have all machines holding all the
data. That doesn't matter much if you're only holding a few gigabytes,
but I can imagine storage requirments escalating (e.g. if we were
offered streaming video of the Hindeburg crash, would we want to turn
it down?)

I don't know how good mySQL is at dealing with this sort of mirroring,
I suspect this may not be the model that its creators had in mind... If
not, what is it? Anyone?

/Bob Dodd







__________________________________________________
Do You Yahoo!?
Yahoo! Auctions - Buy the things you want at great prices. 
http://auctions.yahoo.com/