Re: [rdiff-backup-users] rdiff-backup performance -- slow operation on initial backup?

[Derek Atkins]

Should I be profiling on the backup server or the target/client (the
system being backed up)?
-derek

On Wed, March 30, 2016 1:47 pm, Patrik Dufresne wrote:
> I you want to spent some time debuging this this issue. It might be
> helpful
> to run a profiler.
>
> python -m cProfile -o data.prof /usr/bin/rdiff-backup source destination
>
> Then send us the data.prof. I will use `snakeviz data.prof` to visualize
> the data.
>
> --
> Patrik Dufresne Service Logiciel inc.
> http://www.patrikdufresne.com <http://patrikdufresne.com/>/
> 514-971-6442
> 1-114 rue des Hautbois,
> St-Colomban, QC J5K 2H6
>
> On Wed, Mar 30, 2016 at 1:30 PM, Greg Troxel <address@hidden> wrote:
>
>>
>> "Derek Atkins" <address@hidden> writes:
>>
>> > ... according to repeated netstat the socket buffers are GENERALLY
>> 0,0.
>> I
>> > did see it get up to about 1000, at one point...  OH, I take that
>> back,
>> > the SEND Queue value was just up to 243816 on the target system....
>>
>> If it bursts and comes back, it's probably fine, and many systems
>> dynamically size the buffers.  This is unlikely to be the issue in a LAN
>> environment - more like when you have 100 Mb/s pipes and 70ms latencies.
>>
>> >>> Unfortunately bup is not available on all my target platforms.
>> >>
>> >> bup is python with a little C, and thus seems pretty portable.
>> Where
>> >> isn't it working?
>> >
>> > Didn't say it wasn't working.  I said "it is not available", meaning
>> there
>> > is no prepackaged RPM for some of my systems.  I could certainly try
>> to
>> > piece it together, but of course I prefer to use distro-supplied
>> software
>> > wherever possible.  It makes upgrading much easier.
>>
>> I see.  (It's in pkgsrc, which builds pretty much everywhere, but that
>> gets you into a second packaging system.)
>>
>> >> amanda is basically wrappers around dump and tar.  If you have 50
>> >> machines and want to do level 0/1/2 to tape and take tapes offsite,
>> it
>> >> works great, after you pay for the LTO tape drive.
>> >
>> > I don't have 50 machines.  I do have ~10-15.  Pretty much all Linux
>> boxes.
>> >  Not using tape; backups are all on spinning media.
>>
>> amanda can cope with that (files on disk that are virtual tapes), but
>> I'm not at all sure it's the right approach for you.
>>
>> Good luck figuring out where your bottleneck is.
>>
>> _______________________________________________
>> rdiff-backup-users mailing list at address@hidden
>> https://lists.nongnu.org/mailman/listinfo/rdiff-backup-users
>> Wiki URL:
>> http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki
>>
>


-- 
       Derek Atkins                 617-623-3745
       address@hidden             www.ihtfp.com
       Computer and Internet Security Consultant