[Top][All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [Duplicity-talk] erroneous incrementals? verify failing? everything
From: |
Kenneth Loafman |
Subject: |
Re: [Duplicity-talk] erroneous incrementals? verify failing? everything else seems okay? |
Date: |
Wed, 25 Mar 2009 10:57:20 -0500 |
User-agent: |
Thunderbird 2.0.0.21 (X11/20090318) |
Glad you found it. I should have spotted it earlier.
As to sign key -- you encrypt/sign to your public keys -- then your
private keys are used for decryption and signature verification.
...Ken
Josh wrote:
> OK, looks like it was in my script... I changed some things and I'm
> getting back what I'd expect now...
>
> backup.sh;
> -------
> #!/bin/bash
>
> #
> # Script to backup data to Amazon S3
> #
>
> # Export some ENV variables so you don't have to type anything
> export AWS_ACCESS_KEY_ID=<snip>
> export AWS_SECRET_ACCESS_KEY=<snip>
> export PASSPHRASE=<snip>
>
> # The source of your backup
> SOURCE=/var/www/web2/g2data/albums/cars/jeep
>
> # The destination
> DEST=s3+http://leveleight.test
>
>
> duplicity \
> --encrypt-key=<snip> \
> --sign-key=<snip> \
> ${SOURCE} ${DEST}
>
> # Reset the ENV variables. Don't need them sitting around
> export AWS_ACCESS_KEY_ID=
> export AWS_SECRET_ACCESS_KEY=
> export PASSPHRASE=
> -------
>
> looks like it was in my SOURCE listing, then my exclude and include
> variables... I believe the script I copied was originally intended for a
> complete system backup and I'm only looking to backup a directory... So
> I just made it simpler and it now works...
>
> last question, should I be signing my backups with my public or private key?
>
> thanks for all your help,
> Josh
>
> -------------
> This communication (with attachments) is covered by the U.S.C. Sec.
> 2510-252, is confidential, and intended only for the use of the
> individual or entity named above. If the reader of this message is not
> the intended recipient, you are hereby notified that any dissemination,
> distribution, or copying of this communication is strictly prohibited.
> If you receive this communication in error, notify us immediately,
> return the document(s), and destroy all copies of the original message.
>
>
> On Wed, Mar 25, 2009 at 7:15 AM, Josh <address@hidden
> <mailto:address@hidden>> wrote:
>
> okay, I just tested it with a smaller directory with only 6 photos
> in it, thinking maybe something in my initial upload got borked
> because it was so large or whatever... and it looks like I'm seeing
> the same errors... so I'm thinking its gotta be my scripts... can
> someone take a peek at them for me? here they are...
>
> backup.sh;
> -------
>
> #!/bin/bash
>
> #
> # Script to backup data to Amazon S3
> #
>
> # Export some ENV variables so you don't have to type anything
> export AWS_ACCESS_KEY_ID=<snip>
> export AWS_SECRET_ACCESS_KEY=<snip>
> export PASSPHRASE=<snip>
>
> GPG_KEY=<snip>
>
> # The source of your backup
> SOURCE=/
>
> # The destination
> DEST=s3+http://leveleight.test
>
>
> duplicity \
> --encrypt-key=${GPG_KEY} \
> --sign-key=${GPG_KEY} \
> --include=/var/www/web2/g2data/albums/cars/jeep \
>
> --exclude=/** \
> ${SOURCE} ${DEST}
>
> # Reset the ENV variables. Don't need them sitting around
> export AWS_ACCESS_KEY_ID=
> export AWS_SECRET_ACCESS_KEY=
> export PASSPHRASE=
>
> -------
>
>
> verify.sh;
>
> -------
> #!/bin/bash
>
> # Script to verify data on Amazon S3
>
> export AWS_ACCESS_KEY_ID=<snip>
> export AWS_SECRET_ACCESS_KEY=<snip>
> export PASSPHRASE=<snip>
> GPG_KEY=<snip>
>
> duplicity verify -v9 s3+http://leveleight.test
> /var/www/web2/g2data/albums/cars/jeep
>
>
> # Reset the ENV variables. Don't need them sitting around
> export AWS_ACCESS_KEY_ID=
> export AWS_SECRET_ACCESS_KEY=
> export PASSPHRASE=
> -------
>
>
> here is the output from my verify, which I did _right_ after the
> backup (level 9 verbose);
>
> address@hidden:~/scripts$ ./t_amazon_verify.sh
> Main action: verify
> Listed
>
> s3+http://leveleight.test/duplicity-full-signatures.20090325T120311Z.sigtar.gpg
> Listed
> s3+http://leveleight.test/duplicity-full.20090325T120311Z.manifest.gpg
> Listed
> s3+http://leveleight.test/duplicity-full.20090325T120311Z.vol1.difftar.gpg
> Listed
>
> s3+http://leveleight.test/duplicity-inc.20090325T120311Z.to.20090325T120322Z.manifest.gpg
> Listed
>
> s3+http://leveleight.test/duplicity-inc.20090325T120311Z.to.20090325T120322Z.vol1.difftar.gpg
> Listed
>
> s3+http://leveleight.test/duplicity-new-signatures.20090325T120311Z.to.20090325T120322Z.sigtar.gpg
> 6 files exist on backend
> Extracting backup chains from list of files:
> ['duplicity-full-signatures.20090325T120311Z.sigtar.gpg',
> 'duplicity-full.20090325T120311Z.manifest.gpg',
> 'duplicity-full.20090325T120311Z.vol1.difftar.gpg',
> 'duplicity-inc.20090325T120311Z.to.20090325T120322Z.manifest.gpg',
> 'duplicity-inc.20090325T120311Z.to.20090325T120322Z.vol1.difftar.gpg',
>
> 'duplicity-new-signatures.20090325T120311Z.to.20090325T120322Z.sigtar.gpg']
> File duplicity-full-signatures.20090325T120311Z.sigtar.gpg is not
> part of a known set; creating new set
> Ignoring file (rejected by backup set)
> 'duplicity-full-signatures.20090325T120311Z.sigtar.gpg'
> File duplicity-full.20090325T120311Z.manifest.gpg is not part of a
> known set; creating new set
> File duplicity-full.20090325T120311Z.vol1.difftar.gpg is part of
> known set
> File duplicity-inc.20090325T120311Z.to.20090325T120322Z.manifest.gpg
> is not part of a known set; creating new set
> File
> duplicity-inc.20090325T120311Z.to.20090325T120322Z.vol1.difftar.gpg
> is part of known set
> File
> duplicity-new-signatures.20090325T120311Z.to.20090325T120322Z.sigtar.gpg
> is not part of a known set; creating new set
> Ignoring file (rejected by backup set)
> 'duplicity-new-signatures.20090325T120311Z.to.20090325T120322Z.sigtar.gpg'
> Found backup chain [Wed Mar 25 07:03:11 2009]-[Wed Mar 25 07:03:11 2009]
> Added incremental Backupset (start_time: Wed Mar 25 07:03:11 2009 /
> end_time: Wed Mar 25 07:03:22 2009)
> Added set Wed Mar 25 07:03:22 2009 to pre-existing chain [Wed Mar 25
> 07:03:11 2009]-[Wed Mar 25 07:03:22 2009]
>
> Collection Status
> -----------------
> Connecting with backend: BotoBackend
> Archive dir: None
>
> Found 0 backup chains without signatures.
>
> Found a complete backup chain with matching signature chain:
> -------------------------
> Chain start time: Wed Mar 25 07:03:11 2009
> Chain end time: Wed Mar 25 07:03:22 2009
>
> Number of contained backup sets: 2
> Total number of contained volumes: 2
>
> Type of backup set: Time: Num volumes:
> Full Wed Mar 25 07:03:11 2009 1
> Incremental Wed Mar 25 07:03:22 2009 1
>
> -------------------------
> No orphaned or incomplete backup sets found.
> Last full backup date: Wed Mar 25 07:03:11 2009
> Using temporary directory /tmp/duplicity-y8piFz-tempdir
> Registering (mktemp) temporary file
> /tmp/duplicity-y8piFz-tempdir/mktemp-Avmet--1
> Downloading
> s3+http://leveleight.test/duplicity-full.20090325T120311Z.manifest.gpg
> ===== Begin GnuPG log =====
> gpg: encrypted with 2048-bit ELG-E key, ID <snip>, created <snip>
> "<snip> (Key for Duplicity) <address@hidden
> <mailto:address@hidden>>"
> gpg: Signature made <snip> using DSA key ID <snip>
> gpg: Good signature from "<snip> (Key for Duplicity)
> <address@hidden <mailto:address@hidden>>"
> ===== End GnuPG log =====
> Deleting /tmp/duplicity-y8piFz-tempdir/mktemp-Avmet--1
> Forgetting temporary file /tmp/duplicity-y8piFz-tempdir/mktemp-Avmet--1
> Registering (mktemp) temporary file
> /tmp/duplicity-y8piFz-tempdir/mktemp-TCNn5_-2
> Downloading
> s3+http://leveleight.test/duplicity-full.20090325T120311Z.vol1.difftar.gpg
> Registering (mktemp) temporary file
> /tmp/duplicity-y8piFz-tempdir/mktemp-dCtSlU-3
> Downloading
>
> s3+http://leveleight.test/duplicity-inc.20090325T120311Z.to.20090325T120322Z.manifest.gpg
> ===== Begin GnuPG log =====
> gpg: encrypted with 2048-bit ELG-E key, ID <snip>, created <snip>
> "<snip> (Key for Duplicity) <address@hidden
> <mailto:address@hidden>>"
> gpg: Signature made <snip> using DSA key ID <snip>
> gpg: Good signature from "<snip> (Key for Duplicity)
> <address@hidden <mailto:address@hidden>>"
> ===== End GnuPG log =====
> Deleting /tmp/duplicity-y8piFz-tempdir/mktemp-dCtSlU-3
> Forgetting temporary file /tmp/duplicity-y8piFz-tempdir/mktemp-dCtSlU-3
> Registering (mktemp) temporary file
> /tmp/duplicity-y8piFz-tempdir/mktemp-WV_70M-4
> Downloading
>
> s3+http://leveleight.test/duplicity-inc.20090325T120311Z.to.20090325T120322Z.vol1.difftar.gpg
> ===== Begin GnuPG log =====
> gpg: encrypted with 2048-bit ELG-E key, ID <snip>, created <snip>
> "<snip> (Key for Duplicity) <address@hidden
> <mailto:address@hidden>>"
> gpg: Signature made <snip> using DSA key ID <snip>
> gpg: Good signature from "<snip> (Key for Duplicity)
> <address@hidden <mailto:address@hidden>>"
> ===== End GnuPG log =====
> Deleting /tmp/duplicity-y8piFz-tempdir/mktemp-WV_70M-4
> Forgetting temporary file /tmp/duplicity-y8piFz-tempdir/mktemp-WV_70M-4
> Processed volume 1 of 2
> Selecting /var/www/web2/g2data/albums/cars/jeep
>
> Difference found: File . has permissions 755, expected 755
> Selecting /var/www/web2/g2data/albums/cars/jeep/1.jpg
> Difference found: New file 1.jpg
> Selecting /var/www/web2/g2data/albums/cars/jeep/2.jpg
> Difference found: New file 2.jpg
> Selecting /var/www/web2/g2data/albums/cars/jeep/3.jpg
> Difference found: New file 3.jpg
> Selecting /var/www/web2/g2data/albums/cars/jeep/4.jpg
> Difference found: New file 4.jpg
> Selecting /var/www/web2/g2data/albums/cars/jeep/5.jpg
> Difference found: New file 5.jpg
> Selecting /var/www/web2/g2data/albums/cars/jeep/6.jpg
> Difference found: New file 6.jpg
> Difference found: File var is missing
> Difference found: File var/www is missing
> Difference found: File var/www/web2 is missing
> Difference found: File var/www/web2/g2data is missing
> Difference found: File var/www/web2/g2data/albums is missing
> Difference found: File var/www/web2/g2data/albums/cars is missing
> Difference found: File var/www/web2/g2data/albums/cars/jeep is missing
> Difference found: File var/www/web2/g2data/albums/cars/jeep/1.jpg is
> missing
> Difference found: File var/www/web2/g2data/albums/cars/jeep/2.jpg is
> missing
> Difference found: File var/www/web2/g2data/albums/cars/jeep/3.jpg is
> missing
> Difference found: File var/www/web2/g2data/albums/cars/jeep/4.jpg is
> missing
> Difference found: File var/www/web2/g2data/albums/cars/jeep/5.jpg is
> missing
> Difference found: File var/www/web2/g2data/albums/cars/jeep/6.jpg is
> missing
> ===== Begin GnuPG log =====
> gpg: encrypted with 2048-bit ELG-E key, ID <snip>, created <snip>
> "<snip> (Key for Duplicity) <address@hidden
> <mailto:address@hidden>>"
> gpg: Signature made <snip> using DSA key ID <snip>
> gpg: Good signature from "<snip> (Key for Duplicity)
> <address@hidden <mailto:address@hidden>>"
> ===== End GnuPG log =====
> Deleting /tmp/duplicity-y8piFz-tempdir/mktemp-TCNn5_-2
> Forgetting temporary file /tmp/duplicity-y8piFz-tempdir/mktemp-TCNn5_-2
> Processed volume 2 of 2
> Verify complete: 20 files compared, 20 differences found.
>
> --end---
>
>
> clearly there should be no differences, so what am I doing wrong here?
> thanks in advance,
> Josh
>
>
>
>
> -------------
> This communication (with attachments) is covered by the U.S.C. Sec.
> 2510-252, is confidential, and intended only for the use of the
> individual or entity named above. If the reader of this message is
> not the intended recipient, you are hereby notified that any
> dissemination, distribution, or copying of this communication is
> strictly prohibited. If you receive this communication in error,
> notify us immediately, return the document(s), and destroy all
> copies of the original message.
>
>
> On Wed, Mar 25, 2009 at 6:38 AM, Edgar Soldin <address@hidden
> <mailto:address@hidden>> wrote:
>
>
> > I think the log level should be moved down to 3 (default) on
> this one.
> > Opinions?
> >
>
> Didn't Michael kind of name the log levels or tried at least? ..
> kind of
> warning, info, debug ...
> online man page states
> -->
>
> *-v*/[0-9]/*, --verbosity */[0-9]/
> Specify verbosity level (0 is total silent, 3 is the default,
> and 9
> is noisiest).
> <--
>
> This should be more detailled. If somebody gives me an overview
> I could
> think of a more detailed description for the manpage.
>
> address@hidden .. do the --log-* commands interpret the -v
> switch or
> do they always put out -v9 ?
>
> ..ede
>
> > ...Ken
> >
> > Richard Scott wrote:
> >
> >> Hi,
> >>
> >> I have an active S3 backup routine setup and I've not had
> issues backing up or restoring files.
> >>
> >> However, when I run this:
> >>
> >> # /usr/bin/duplicity verify -v 9
> --archive-dir=/var/lib/duplicity/etc
> >> s3+http://uniquefoldername/etc /etc
> >> Verify complete: 1162 files compared, 1 difference found.
> >>
> >> I don't know how to display what the difference is?
> >>
> >> I get vary little output... I thought the "-v 9" would turn
> on verbose output?
> >>
> >> I have the following installed:
> >>
> >> app-backup/duplicity-0.5.12
> >> dev-lang/python-2.4.3-r4
> >> dev-lang/python-2.5.2-r7
> >> dev-python/boto-1.6b
> >> dev-python/py-gnupg-0.3.2
> >> dev-python/pycrypto-2.0.1-r6
> >>
> >> # python -V
> >> Python 2.5.2
> >>
> >> Ken, is there any commands I can test or check for you?
> >>
> >> Rich.
> >>
> >>
> >>> It turns out we're on the same system across the board. I
> thought I had
> >>> not upgraded to 8.04.2, but it looks like I did. 32-bit
> here as well.
> >>>
> >>> Not sure what's going on. If you get the chance, try a
> restore and make
> >>> sure its OK. I just don't see any problem from here. I do
> want to find it though, so maybe
> >>> someone else can chip in.
> >>>
> >>> ...Ken
> >>>
> >>
> >> _______________________________________________
> >> Duplicity-talk mailing list
> >> address@hidden <mailto:address@hidden>
> >> http://lists.nongnu.org/mailman/listinfo/duplicity-talk
> >>
> >>
> >
> >
> >
> >
>
> ------------------------------------------------------------------------
> >
> > _______________________________________________
> > Duplicity-talk mailing list
> > address@hidden <mailto:address@hidden>
> > http://lists.nongnu.org/mailman/listinfo/duplicity-talk
> >
>
>
>
> _______________________________________________
> Duplicity-talk mailing list
> address@hidden <mailto:address@hidden>
> http://lists.nongnu.org/mailman/listinfo/duplicity-talk
>
>
>
>
> ------------------------------------------------------------------------
>
> _______________________________________________
> Duplicity-talk mailing list
> address@hidden
> http://lists.nongnu.org/mailman/listinfo/duplicity-talk
signature.asc
Description: OpenPGP digital signature
- Re: [Duplicity-talk] erroneous incrementals? verify failing? everything else seems okay?, (continued)
- Re: [Duplicity-talk] erroneous incrementals? verify failing? everything else seems okay?, Kenneth Loafman, 2009/03/25
- Loglevels and Output ... WAS: Re: [Duplicity-talk] erroneous incrementals?, Edgar Soldin, 2009/03/25
- Re: Loglevels and Output ... WAS: Re: [Duplicity-talk] erroneous incrementals?, Michael Terry, 2009/03/27
- Re: Loglevels and Output ... WAS: Re: [Duplicity-talk] erroneous incrementals?, edgar . soldin, 2009/03/30
- Re: Loglevels and Output ... WAS: Re: [Duplicity-talk] erroneous incrementals?, Michael Terry, 2009/03/30
- Re: Loglevels and Output ... WAS: Re: [Duplicity-talk] erroneous incrementals?, Kenneth Loafman, 2009/03/30
- Re: Loglevels and Output ... WAS: Re: [Duplicity-talk] erroneous incrementals?, Michael Terry, 2009/03/30
- Re: Loglevels and Output ... WAS: Re: [Duplicity-talk] erroneous incrementals?, Kenneth Loafman, 2009/03/30
- Re: [Duplicity-talk] erroneous incrementals? verify failing? everything else seems okay?, Josh, 2009/03/25
- Re: [Duplicity-talk] erroneous incrementals? verify failing? everything else seems okay?, Josh, 2009/03/25
- Re: [Duplicity-talk] erroneous incrementals? verify failing? everything else seems okay?,
Kenneth Loafman <=