Re: [Duplicity-talk] erroneous incrementals? verify failing? everything

On Tue, Mar 24, 2009 at 3:30 PM, Kenneth Loafman <address@hidden> wrote:

I just ran a 53-volume backup of .jpg files to S3 and verified. No
problems found. I'm on 8.04.1 with Boto 1.61, and duplicity 0.5.12.
That should be a close match to your system.

What backup command line did you use?

Are you 32-bit or 64-bit?

...Ken

Josh wrote:
> Its on a Linux system.
>
> The data is mounted via the root partition (within "/" path)
>
> ext3 format
>
> -------------
> This communication (with attachments) is covered by the U.S.C. Sec.
> 2510-252, is confidential, and intended only for the use of the
> individual or entity named above. If the reader of this message is not
> the intended recipient, you are hereby notified that any dissemination,
> distribution, or copying of this communication is strictly prohibited.
> If you receive this communication in error, notify us immediately,
> return the document(s), and destroy all copies of the original message.
>
>
> On Tue, Mar 24, 2009 at 12:00 PM, Kenneth Loafman <address@hidden

> <mailto:address@hidden>> wrote:
>

> Don't bother finishing... I'm going to have to test some against S3
> myself. I just don't see what's going on.
>
> What about the other questions?
>
> Is the data located on a Windows system or a Linux system?
>
> Is it mounted, if so how?
>
> I'm guessing its on the Linux system directly from the path, but I just
> want to verify that. Do you know what kind of filesystem its on?
>
> ...Ken
>
> Josh wrote:
> > here is the scrip I am using for the verify;
> >
> > -------------
> > #!/bin/bash
> >
> > # Script to verify data on Amazon S3
> >
> > export AWS_ACCESS_KEY_ID=<key>
> > export AWS_SECRET_ACCESS_KEY=<key>
> > export PASSPHRASE=<phrase>
> > GPG_KEY=<key>
> >
> > duplicity verify -v9 s3+http://leveleight.gallery
> > /var/www/web2/g2data/albums/
> >
> > # Reset the ENV variables. Don't need them sitting around
> > export AWS_ACCESS_KEY_ID=
> > export AWS_SECRET_ACCESS_KEY=
> > export PASSPHRASE=
> > -------------
> >
> > here are some lines from when I run this script, that may/not be of
> > use... I still haven't let it run completely as its taking a so
> long...
> > but here is what I have
> >
> > Main action: verify
> > Listed
> >
> s3+http://leveleight.gallery/duplicity-full-signatures.2009-03-23T13:27:46-05:00.sigtar.gpg
> > Listed
> >
> s3+http://leveleight.gallery/duplicity-full.2009-03-23T13:27:46-05:00.manifest.gpg
> > Listed
> >
> s3+http://leveleight.gallery/duplicity-full.2009-03-23T13:27:46-05:00.vol1.difftar.gpg
> > Listed
> >
> s3+http://leveleight.gallery/duplicity-full.2009-03-23T13:27:46-05:00.vol10.difftar.gpg
> > Listed
> >
> s3+http://leveleight.gallery/duplicity-full.2009-03-23T13:27:46-05:00.vol100.difftar.gpg
> > Listed
> >
> s3+http://leveleight.gallery/duplicity-full.2009-03-23T13:27:46-05:00.vol1000.difftar.gpg
> > Listed
> >
> s3+http://leveleight.gallery/duplicity-full.2009-03-23T13:27:46-05:00.vol1001.difftar.gpg
> > Listed
> >
> s3+http://leveleight.gallery/duplicity-full.2009-03-23T13:27:46-05:00.vol1002.difftar.gpg
> > Listed
> >
> s3+http://leveleight.gallery/duplicity-full.2009-03-23T13:27:46-05:00.vol1003.difftar.gpg
> > Listed
> >
> s3+http://leveleight.gallery/duplicity-full.2009-03-23T13:27:46-05:00.vol1004.difftar.gpg
> > Listed
> >
> s3+http://leveleight.gallery/duplicity-full.2009-03-23T13:27:46-05:00.vol1005.difftar.gpg
> > Listed
> >
> s3+http://leveleight.gallery/duplicity-full.2009-03-23T13:27:46-05:00.vol1006.difftar.gpg
> > Listed
> >
> s3+http://leveleight.gallery/duplicity-full.2009-03-23T13:27:46-05:00.vol1007.difftar.gpg
> > Listed
> >
> s3+http://leveleight.gallery/duplicity-full.2009-03-23T13:27:46-05:00.vol1008.difftar.gpg
> > Listed
> >
> s3+http://leveleight.gallery/duplicity-full.2009-03-23T13:27:46-05:00.vol1009.difftar.gpg
> > Listed
> >
> s3+http://leveleight.gallery/duplicity-full.2009-03-23T13:27:46-05:00.vol101.difftar.gpg
> > Listed
> >
> s3+http://leveleight.gallery/duplicity-full.2009-03-23T13:27:46-05:00.vol1010.difftar.gpg
> > Listed
> >
> s3+http://leveleight.gallery/duplicity-full.2009-03-23T13:27:46-05:00.vol1011.difftar.gpg
> > Listed
> >
> s3+http://leveleight.gallery/duplicity-full.2009-03-23T13:27:46-05:00.vol1012.difftar.gpg
> > Listed
> >
> s3+http://leveleight.gallery/duplicity-full.2009-03-23T13:27:46-05:00.vol1013.difftar.gpg
> > Listed
> >
> s3+http://leveleight.gallery/duplicity-full.2009-03-23T13:27:46-05:00.vol1014.difftar.gpg
> > Listed
> >
> s3+http://leveleight.gallery/duplicity-full.2009-03-23T13:27:46-05:00.vol1015.difftar.gpg
> >
> > <snip>
> >
> > Listed
> >
> s3+http://leveleight.gallery/duplicity-inc.20090323T182746Z.to.20090324T122905Z.manifest.gpg
> > Listed
> >
> s3+http://leveleight.gallery/duplicity-inc.20090323T182746Z.to.20090324T122905Z.vol1.difftar.gpg
> > Listed
> >
> s3+http://leveleight.gallery/duplicity-new-signatures.20090323T182746Z.to.20090324T122905Z.sigtar.gpg
> > 1405 files exist on backend
> >
> > <snip>
> >
> > Extracting backup chains from list of files:
> > ['duplicity-full-signatures.2009-03-23T13:27:46-05:00.sigtar.gpg',
> > 'duplicity-full.2009-03-23T13:27:46-05:00.manifest.gpg',
> > 'duplicity-full.2009-03-23T13:27:46-05:00.vol1.difftar.gpg',
> > 'duplicity-full.2009-03-23T13:27:46-05:00.vol10.difftar.gpg',
> > 'duplicity-full.2009-03-23T13:27:46-05:00.vol100.difftar.gpg',
> > 'duplicity-full.2009-03-23T13:27:46-05:00.vol1000.difftar.gpg',
> > 'duplicity-full.2009-03-23T13:27:46-05:00.vol1001.difftar.gpg',
> > 'duplicity-full.2009-03-23T13:27:46-05:00.vol
> >
> > <snip>
> >
> > File duplicity-full-signatures.2009-03-23T13:27:46-05:00.sigtar.gpg is
> > not part of a known set; creating new set
> > Ignoring file (rejected by backup set)
> > 'duplicity-full-signatures.2009-03-23T13:27:46-05:00.sigtar.gpg'
> > File duplicity-full.2009-03-23T13:27:46-05:00.manifest.gpg is not part
> > of a known set; creating new set
> > File duplicity-full.2009-03-23T13:27:46-05:00.vol1.difftar.gpg is part
> > of known set
> > File duplicity-full.2009-03-23T13:27:46-05:00.vol10.difftar.gpg is
> part
> > of known set
> >
> > <snip>
> >
> > ile duplicity-full.2009-03-23T13:27:46-05:00.vol999.difftar.gpg is
> part
> > of known set
> > File
> duplicity-inc.20090323T182746Z.to.20090324T122905Z.manifest.gpg is
> > not part of a known set; creating new set
> > File
> duplicity-inc.20090323T182746Z.to.20090324T122905Z.vol1.difftar.gpg
> > is part of known set
> > File
> >
> duplicity-new-signatures.20090323T182746Z.to.20090324T122905Z.sigtar.gpg
> > is not part of a known set; creating new set
> > Ignoring file (rejected by backup set)
> >
> 'duplicity-new-signatures.20090323T182746Z.to.20090324T122905Z.sigtar.gpg'
> > Found backup chain [Mon Mar 23 13:27:46 2009]-[Mon Mar 23 13:27:46
> 2009]
> > Added incremental Backupset (start_time: Mon Mar 23 13:27:46 2009 /
> > end_time: Tue Mar 24 07:29:05 2009)
> > Added set Tue Mar 24 07:29:05 2009 to pre-existing chain [Mon Mar 23
> > 13:27:46 2009]-[Tue Mar 24 07:29:05 2009]
> > Collection Status
> > -----------------
> > Connecting with backend: BotoBackend
> > Archive dir: None
> >
> > Found 0 backup chains without signatures.
> >
> > Found a complete backup chain with matching signature chain:
> > -------------------------
> > Chain start time: Mon Mar 23 13:27:46 2009
> > Chain end time: Tue Mar 24 07:29:05 2009
> > Number of contained backup sets: 2
> > Total number of contained volumes: 1401
> > Type of backup set: Time: Num
> volumes:
> > Full Mon Mar 23 13:27:46 2009
> 1400
> > Incremental Tue Mar 24 07:29:05 2009
> 1
> > -------------------------
> > No orphaned or incomplete backup sets found.
> > Last full backup date: Mon Mar 23 13:27:46 2009
> > Using temporary directory /tmp/duplicity-rZ55N_-tempdir
> > Registering (mktemp) temporary file
> > /tmp/duplicity-rZ55N_-tempdir/mktemp-jcGOJX-1
> > Downloading
> >
> s3+http://leveleight.gallery/duplicity-full.2009-03-23T13:27:46-05:00.manifest.gpg
> > ===== Begin GnuPG log =====
> > gpg: encrypted with 2048-bit ELG-E key, ID BLAHLAH, created BLAHLAH
> > "BLAHLAH (Key for Duplicity) <address@hidden
> <mailto:address@hidden>

> > <mailto:address@hidden <mailto:address@hidden>>>"

> > gpg: Signature made BLAHLAH using DSA key ID BLAHLAH
> > gpg: Good signature from "BLAHLAH (Key for Duplicity)
> > <address@hidden <mailto:address@hidden>

> <mailto:address@hidden <mailto:address@hidden>>>"

> > ===== End GnuPG log =====
> > Deleting /tmp/duplicity-rZ55N_-tempdir/mktemp-jcGOJX-1
> > Forgetting temporary file
> /tmp/duplicity-rZ55N_-tempdir/mktemp-jcGOJX-1
> > Registering (mktemp) temporary file
> > /tmp/duplicity-rZ55N_-tempdir/mktemp-BUaHLW-2
> > Downloading
> >
> s3+http://leveleight.gallery/duplicity-full.2009-03-23T13:27:46-05:00.vol1.difftar.gpg
> > Registering (mktemp) temporary file
> > /tmp/duplicity-rZ55N_-tempdir/mktemp-YhiFhu-3
> > Downloading
> >
> s3+http://leveleight.gallery/duplicity-inc.20090323T182746Z.to.20090324T122905Z.manifest.gpg
> > ===== Begin GnuPG log =====
> >
> > <snip>
> >
> > Forgetting temporary file
> /tmp/duplicity-rZ55N_-tempdir/mktemp-YhiFhu-3
> > Registering (mktemp) temporary file
> > /tmp/duplicity-rZ55N_-tempdir/mktemp-LXMvsD-4
> > Downloading
> >
> s3+http://leveleight.gallery/duplicity-inc.20090323T182746Z.to.20090324T122905Z.vol1.difftar.gpg
> > Selecting /var/www/web2/g2data/albums
> > Difference found: File . has permissions 755, expected 755
> > Selecting /var/www/web2/g2data/albums/2005
> > Difference found: New file 2005
> > Selecting /var/www/web2/g2data/albums/2005/08-11-05_1137.jpg
> > Difference found: New file 2005/08-11-05_1137.jpg
> > Selecting /var/www/web2/g2data/albums/2005/08-12-05_1217.jpg
> >
> >
> >
> >
> >
> >
> >
> > -------------
> > This communication (with attachments) is covered by the U.S.C. Sec.
> > 2510-252, is confidential, and intended only for the use of the
> > individual or entity named above. If the reader of this message is not
> > the intended recipient, you are hereby notified that any
> dissemination,
> > distribution, or copying of this communication is strictly prohibited.
> > If you receive this communication in error, notify us immediately,
> > return the document(s), and destroy all copies of the original
> message.
> >
> >
> > On Tue, Mar 24, 2009 at 10:16 AM, Kenneth Loafman
> <address@hidden <mailto:address@hidden>

> > <mailto:address@hidden <mailto:address@hidden>>> wrote:
> >
> > Josh wrote:
> > > I think you misunderstood, I am currently at;
> > >
> > > Ubuntu version 8.04.2
> > > duplicity version 0.5.12
> > > boto version 1.6a
> >
> > Yes, I did misunderstand. Let's start this over...
> >
> > Could you show me the command lines you used?
> >
> > Is the data located on a Windows system or a Linux system?
> >
> > Is it mounted, if so how?
> >
> > It would also help if you could run the verify with a -v9 and
> let us see
> > the first 200 lines and the last 200 lines of the log. The
> rest is
> > really not needed.
> >
> > ...Thanks,
> > ...Ken
> >
> >
> > _______________________________________________
> > Duplicity-talk mailing list
> > address@hidden <mailto:address@hidden>

> <mailto:address@hidden <mailto:address@hidden>>

> > http://lists.nongnu.org/mailman/listinfo/duplicity-talk
> >
> >
> >
> >
> ------------------------------------------------------------------------
> >
> > _______________________________________________
> > Duplicity-talk mailing list
> > address@hidden <mailto:address@hidden>
> > http://lists.nongnu.org/mailman/listinfo/duplicity-talk
>
>
>
> _______________________________________________
> Duplicity-talk mailing list
> address@hidden <mailto:address@hidden>
> http://lists.nongnu.org/mailman/listinfo/duplicity-talk
>
>
>
> ------------------------------------------------------------------------
>
> _______________________________________________
> Duplicity-talk mailing list
> address@hidden
> http://lists.nongnu.org/mailman/listinfo/duplicity-talk

_______________________________________________
Duplicity-talk mailing list
address@hidden
http://lists.nongnu.org/mailman/listinfo/duplicity-talk

From:	Josh
Subject:	Re: [Duplicity-talk] erroneous incrementals? verify failing? everything else seems okay?
Date:	Tue, 24 Mar 2009 15:37:37 -0500