Re: [rdiff-backup-users] Throttling

[Ben Escoto]

>>>>> "DS" == Dave Steinberg <address@hidden>
>>>>> wrote the following on Sat, 21 Jun 2003 14:11:42 -0400

  DS> Hello Ben and list, In an effort to keep my bandwidth costs
  DS> down, I've been investigating using 'throttle', a bandwidth
  DS> limiting pipe, to slow down rdiff-backup.  It copies stdin to
  DS> stdout and at most as fast as the specified rate.  Some odd
  DS> notes:

  DS> (1) I know about --sleep-ratio, but it doesn't actually help me
  DS> due to the way I am billed.  I am billed on 95% peak usage,
  DS> ergo, if rdiff-backup uses 1600kbps every 30 seconds, that
  DS> doesn't help me as much as if it used 100 kbps 100% of the time.
  DS> The former would cause my bills to go up, the latter would not.
  DS> If I was billed on average usage, then --sleep-ration would
  DS> suffice.

Good, I'm glad --sleep-ratio doesn't help because that option has been
removed.  :-)

  (2) I've tried playing with --remote-schema as such:
  DS> --remote-schema 'ssh %s "throttle -k 100 | rdiff-backup
  DS> --server"' And --remote-schema 'ssh %s "rdiff-backup --server |
  DS> throttle -k 100"'

  DS> But neither work.  Using --verbosity 9, I see that it
  DS> establishes the connection and then just hangs; presumably
  DS> throttle is waiting for input on stdin.

  DS> Could anyone comment whether this behavior is impossible or if
  DS> I've just mucked up the details somewhere?

Should work in principle.  Perhaps the problem is with throttle (it's
not flushing data right?).  In theory you should use throttle on both
ends, to limit traffic in both directions (assuming you are charged
that way).  On my system, this works:

    ~ $ rdiff-backup --test-server --remote-schema %s 'rdiff-backup 
--server'::foo
    Testing server started by:  rdiff-backup --server
    Server OK

But that's just a sanity check.  However, this also works:

    ~ $ rdiff-backup --test-server --remote-schema %s 'cat | rdiff-backup 
--server | cat'::foo
    Testing server started by:  cat | rdiff-backup --server | cat
    Server OK

which seems to show that you can a pipe on both ends.  Next thing to
try would be:

    rdiff-backup --test-server --remote-schema %s 'throttle -k 100 | 
rdiff-backup --server | throttle -k 100'::foo

but I don't have throttle on my machine.  (I think someone else did
this with cstream though, which is similar, maybe check the mailing
list.)


It seems that bandwidth limits should be managed by the OS.  Failing
that, ssh should do it.  Failing that, someone should come up with a
general purpose port forwarder which also limits bandwidth.  Something
like this could work with all applications, not just rdiff-backup or
other apps that use pipes.


-- 
Ben Escoto

pgp3K2PJ9SMoe.pgp
Description: PGP signature