gluster-devel
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Gluster-devel] Re: AFR load-balancing


From: Kevan Benson
Subject: Re: [Gluster-devel] Re: AFR load-balancing
Date: Tue, 20 Nov 2007 12:25:45 -0800
User-agent: Thunderbird 2.0.0.9 (X11/20071031)

Krishna Srinivas wrote:
On Nov 21, 2007 1:05 AM, Anand Avati <address@hidden> wrote:

Have you tried chaining AFR volumes?  There's quite a few ways I can
imagine reducing line saturation if that's the problem.  Here's one:

server1 has an afr of a local volume and a volume from server2

server3 has an afr of a local volume and a volume from server4

client afr of server1's afr and server2's afr

This should allow a single write from a client at 1000/8/2 = 62.5
MByte/s.  Client writes only twice (to server1 and server3) halving the
bandwidth.  Server1 and server3 write only once each to server2 and
server4 respectively and receive the writes from the client, halving
their bandwidth.

Note: I have no idea how well this will perform in reality.  There may
be enough lag in glusterfs chaining writes that the gains aren't worth
it, but I suspect since it is effectively pipelining the writes along
there won't be too much lag.

Good thought kevan, it sounds like a good in-between solution.
Jerker, this might work even better for you if the server-server
interconnects are on a seperate physical media not cascaded with the
server-client interconnect.

True. Actually if afr is on the server side, write performance will
increase only
if the server-server interconnect is on a different network (or VLAN)
than the server-client network. If they are on same LAN, regardless
of where the AFR is loaded (either client or server) there should be
very little difference in performance (atleast theoretically) because
even if AFR is loaded on the server it will still have to do the writes
to all the other servers.

Yeah, that's why I was careful to state that the client should AFR the servers it connects to directly. That way, each server has at most two network operations per file operation.

You could, theoretically, get gigabit performance if every system in the glusterfs cluster has two nics, and you have switch ports, vlans and private netblocks to spare.

E.g.
Client[ETH0]->Server(N)[ETH0],Server(N)[ETH1]->Server(N+1)[ETH0],..,Server((N/2)-1)[ETH1]->Server(N/2)[ETH0]
Client[ETH1]->Server((N/2)+1)[ETH0],Server((N/2)+1)[ETH1]->Server((N/2)+2)[ETH0],..,Server(N-1)[ETH1]->Server(N)[ETH0]

I suspect adding additional servers to the chain beyond 1 or 2 would introduce some serious lag, and thus make it not very useful. You would also be taxing the backplane of most switches fairly quickly if you used vlans and few switches.

--

-Kevan Benson
-A-1 Networks




reply via email to

[Prev in Thread] Current Thread [Next in Thread]