Re: [Sks-devel] De-peering (temp) very stale peers

[Phil Pennock]

On 2013-02-13 at 08:37 -0400, Jason Harris wrote:
> On Wed, Feb 13, 2013 at 01:06:40AM -0500, Phil Pennock wrote:
> > Does anyone have any figures for the load caused by significant
> > differences in the numbers of keys had by two keyservers and how quickly
> > SKS stops being able to cope efficiently?
> 
> > A couple of my peers are >100K keys behind, so I've temporarily
> 
> Figuring out the differences in the working set happens quickly enough.
> But then your server will spin its wheels grabbing all those 100k keys
> by their old hashes and parsing every last one just to find that there
> are no new packets in there.
> 
> Your peer will be pulling those same keys, via their newer hashes,
> in blocks of 100 keys (default setting).  If your server and the
> communications to your peer are fast, that will minimize the non-
> multitasking time SKS spends grabbing the keys and transmitting them.
> An HTTP proxy will let "sks db" get back to work sooner, instead of
> blocking until it finishes transmitting the data to the peer.

My take away from this is that the SKS reconciliation protocol needs to
let each server state up front how many keys they have, and the one with
"more" keys, if the difference is greater than a tunable (1000) should
ignore the set difference and not request the keys from the peer.

Once the peer has roughly the same number of keys, any remaining set
difference results from the peers having disjoint sets, and
reconciliation can work as normal to resolve the differences, on the
next reconciliation attempt between those two servers.

It seems that this would prevent the worst of the remaining impact
(after the front-end proxy on 11371, which I didn't have back then) for
the up-to-date peer, and the worst case impact is that keys which have
only been updated on the not-up-to-date peer won't make it out until
that peer is mostly otherwise up-to-date.

The rest of the impact on the up-to-date peer is then just bandwidth and
bulk key retrieval.

Am I missing something?  Does this seem like a sensible protocol update
to consider?

-Phil

pgp1rPPjUo6M8.pgp
Description: PGP signature