[Duplicity-talk] What is Duplicity doing during incremental backup?

[Nicholas Sherlock]

Hi everyone,

I'm trying to backup a copy of my database to RackSpace cloud file storage using duplicity 0.6.15. The initial full backup went fine:

Local and Remote metadata are synchronized, no sync needed.

Last full backup date: none

No signatures found, switching to full backup.

--------------[ Backup Statistics ]--------------

StartTime 1329829385.54 (Tue Feb 21 13:03:05 2012)

EndTime 1329851993.83 (Tue Feb 21 19:19:53 2012)

ElapsedTime 22608.29 (6 hours 16 minutes 48.29 seconds)

SourceFiles 797

SourceFileSize 96807374962 (90.2 GB)

NewFiles 797

NewFileSize 96807374962 (90.2 GB)

DeletedFiles 0

ChangedFiles 0

ChangedFileSize 0 (0 bytes)

ChangedDeltaSize 0 (0 bytes)

DeltaEntries 797

RawDeltaSize 96807325776 (90.2 GB)

TotalDestinationSizeChange 30583984834 (28.5 GB)

Errors 0

-------------------------------------------------

Now I'm trying to re-run duplicity in order to create an incremental backup:

duplicity --allow-source-mismatch --volsize 200 /db cf+http://${CLOUDFILES_CONTAINER}

Local and Remote metadata are synchronized, no sync needed.

Last full backup date: Tue Feb 21 13:03:05 2012

However, it has been running for a long time and doesn't seem to be doing very much. No new files from the incremental backup have appeared in Rackspace. iostat shows that nothing is happening on disk (nothing is being read from the source device, /dev/md127):

# iostat 60

Linux 3.0.0-14-virtual (domU-12-31-38-04-24-66)         02/22/2012      _i686_  (1 CPU)

avg-cpu:  %user   %nice %system %iowait  %steal   %idle

           0.81    0.00   35.14    0.24   54.12    9.69

Device:            tps    kB_read/s    kB_wrtn/s    kB_read    kB_wrtn

md127             0.47        15.64         0.36     135998       3162

xvdf              0.10         3.90         0.09      33924        800

xvdg              0.11         3.90         0.09      33891        808

xvdh              0.16         3.96         0.09      34462        776

xvdi              0.11         3.90         0.09      33903        778

avg-cpu:  %user   %nice %system %iowait  %steal   %idle

           0.00    0.00   39.89    0.00   60.11    0.00

Device:            tps    kB_read/s    kB_wrtn/s    kB_read    kB_wrtn

md127             0.00         0.00         0.00          0          0

xvdf              0.00         0.00         0.00          0          0

xvdg              0.00         0.00         0.00          0          0

xvdh              0.00         0.00         0.00          0          0

xvdi              0.00         0.00         0.00          0          0

But despite this, Duplicity seems to be very busy doing something, using 99.5% of the CPU and 86.7% of the memory, looks like it has been running for 2 hours now:

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND

 1129 root      20   0 1455m 1.4g 4716 R 99.5 86.7 126:05.34 duplicity

Is that normal? What the heck is it doing? Could I speed up this process by running it on a multi-core box?

Cheers,

Nicholas Sherlock