why didn't you simply set PATH=/usr/local/bin:/usr/bin:/bin in cron?
Yes, I could have done that.
If I manually instantiate a cron-like sh I see:
sh-4.1# env
SHELL=/bin/sh
USER=root
PATH=/usr/bin:/bin
PWD=/root
LANG=en_US.UTF-8
HOME=/root
SHLVL=2
LOGNAME=root
_=/usr/bin/env
sh-4.1# python --version
Python 2.6.6
sh-4.1# duplicity --version
duplicity 0.7.12
However this is with the modified duplicity script where I replaced #!/usr/bin/env python (default 2.6.6 that ships with CentOS) with #!/usr/bin/env /usr/local/bin/python (my custom 2.7.13 version)
Anyways, Last night all duply cron jobs ran without a hitch and they all used the specified .cache directory instead of /root/.cache
Clearly I've messed things up by mixing 2 different python versions on my system and naming the new python the same as the old python.
duplicity has apparently only been installed into /usr/lib/python2.7/site-packages/ and NOT into /usr/lib/python2.6/site-packages/
I think I understand a bit better now how duplicity and duply interact with python and I should be able to clean things up. Thanks for your patience, and thanks for the development work on duply/duplicity.
can you send me the output from 'duplicity --version', when run with the old python 2.6 please?..ede/duply.netOn 10.04.2017 20:42, Scott Classen wrote:Hello
OK. I think the root (pun intended) of the problem is that I have both the distro python and my custom python installed.
The cron env is:
SHELL=/bin/sh
USER=root
PATH=/usr/bin:/bin
PWD=/root
LANG=en_US.UTF-8
SHLVL=1
HOME=/root
LOGNAME=root
The duplicity script uses:
#!/usr/bin/env python
which, in the limited cron environment is picking up python 2.6 (the default system version).
I've changed this to:
#!/usr/bin/env /usr/local/bin/python
which now uses my custom installed python 2.7.13
I'm hoping that this will fix the cache issue.
On Apr 10, 2017, at 10:31 AM, edgar.soldin--- via Duplicity-talk <address@hidden <mailto:address@hidden>> wrote:
Scott,
1.
i need the _complete_ output of the command.
This is the full output of the command that I run in crontab.
sh-4.1# /usr/bin/duply /etc/duply/home_a pre_full_purgeFull_post --force --preview
Start duply v2.0.1, time is 2017-04-10 11:20:25.
Using profile '/etc/duply/home_a'.
Using installed duplicity version 0.7.12, python 2.7.13, gpg 2.0.14 (Home: ~/.gnupg), awk 'GNU Awk 3.1.7', grep 'grep (GNU grep) 2.20', bash '4.1.2(2)-release (x86_64-redhat-linux-gnu)'.
Autoset found secret key of first GPG_KEY entry 'ABCDEFG' for signing.
-- Run cmd -- Checking TEMP_DIR '/duplicity' is a folder and writable --
test -d '/duplicity' && test -w '/duplicity' 2>&1
-- Run cmd -- Test - Encrypt to 'ABCDEFG' & Sign with 'ABCDEFG' --
echo password123 | gpg --sign --default-key ABCDEFG --passphrase-fd 0 --batch -r ABCDEFG --status-fd 1 -o '/duplicity/duply.23401.1491848425_ENC' -e '/usr/bin/duply' 2>&1
-- Run cmd -- Test - Decrypt --
echo password123 | gpg --passphrase-fd 0 --batch -o '/duplicity/duply.23401.1491848425_DEC' -d '/duplicity/duply.23401.1491848425_ENC' 2>&1
-- Run cmd -- Test - Compare --
test "$(cat '/usr/bin/duply')" = "$(cat '/duplicity/duply.23401.1491848425_DEC')" 2>&1
Cleanup - Delete '/duplicity/duply.23401.1491848425_*'(FAILED)
--- Start running command PRE at 11:20:25.775 ---
/etc/duply/home_a/pre
--- Finished state OK at 11:20:25.789 - Runtime 00:00:00.013 ---
--- Start running command FULL at 11:20:25.799 ---
TMPDIR='/duplicity' PASSPHRASE=password123 FTP_PASSWORD='PASSWORD' '/usr/local/bin/python' '/usr/bin/duplicity' full --archive-dir '/duplicity/.duply-cache' --name duply_home_a --encrypt-key ABCDEFG --sign-key ABCDEFG --verbosity '4' --volsize 500 --s3-use-rrs --asynchronous-upload --force --exclude-filelist '/etc/duply/home_a/exclude' '/home' 's3://amazonaws.com/swq/a'
--- Finished state OK at 11:20:25.847 - Runtime 00:00:00.048 ---
--- Start running command PURGEFULL at 11:20:25.857 ---
TMPDIR='/home-old/duplicity' PASSPHRASE=password123 FTP_PASSWORD='PASSWORD' '/usr/local/bin/python' '/usr/bin/duplicity' remove-all-but-n-full 4 --archive-dir '/home-old/duplicity/.duply-cache' --name duply_home_a --encrypt-key ABCDEFG --sign-key ABCDEFG --verbosity '4' --volsize 500 --s3-use-rrs --asynchronous-upload --force 's3://amazonaws.com/swq/a'
--- Finished state OK at 11:20:25.885 - Runtime 00:00:00.028 ---
--- Start running command POST at 11:20:25.896 ---
/etc/duply/home_a/post
--- Finished state OK at 11:20:25.910 - Runtime 00:00:00.014 ---
2.
additionally duply complains that it cannot parse your version
INFO:
duplicity version check failed (please report, this is a bug)
Using profile '/etc/duply/home_a'.
Using installed duplicity version MISSING,
that is unexpected.. ede
On 10.04.2017 18:36, Scott Classen wrote:
All my backup jobs are executed as cron jobs.
I just had a look at one of the log files from last night and I'm seeing the following at the top of the log file:
# cat /data/log/duply/home_a_incr_2017-04-10.log
Start duply v2.0.1, time is 2017-04-10 01:00:01.
INFO:
duplicity version check failed (please report, this is a bug)
Using profile '/etc/duply/home_a'.
Using installed duplicity version MISSING, python 2.7.13, gpg 2.0.14 (Home: ~/.gnupg), awk 'GNU Awk 3.1.7', grep 'grep (GNU grep) 2.20', bash '4.1.2(2)-release (x86_64-redhat-linux-gnu)'.
Autoset found secret key of first GPG_KEY entry '78ABCDEF' for signing.
Checking TEMP_DIR '/bup/duplicity' is a folder and writable (OK)
Test - Encrypt to '78ABCDEF' & Sign with '78ABCDEF' (OK)
Test - Decrypt (OK)
Test - Compare (OK)
Cleanup - Delete '/bup/duplicity/duply.30881.1491811201_*'(OK)
Perhaps there is something wrong with the environment variables for duply jobs run in a cron setting? The backup jobs are completing successfully. The main problem at the moment is that the cache files aren't going where they are supposed to.
Cheers,
Scott
On Apr 9, 2017, at 6:09 PM, Scott Classen <address@hidden <mailto:address@hidden> <mailto:address@hidden>> wrote:
On Apr 9, 2017, at 11:09 AM, edgar.soldin--- via Duplicity-talk <address@hidden <mailto:address@hidden> <mailto:address@hidden>> wrote:
On 09.04.2017 19:52, Scott Classen via Duplicity-talk wrote:
Hello,
I have a dozen or so duply profiles. All of them have the cache directory specified as a separate volume on my main backup machine.
e.g.
# default '~/.cache/duplicity/duply_<profile>/'
# if set '${ARCH_DIR}/<profile>'
#ARCH_DIR=/some/space/safe/.duply-cache
ARCH_DIR=/bup/duplicity/.duply-cache
However, it appears that duply/duplicity is not honoring this configuration directive and all of the cache files are are going into /root/.cache/duplicity/
This is causing me severe headaches as the root partition is getting critically full.
hi Scott,
1.
what's your duply version?
2.0.1
2.
please run one backup w/ the '--preview' parameter and post it's result. check for and obfuscate private strings beforehand.
[root]# duply project status --preview
Start duply v2.0.1, time is 2017-04-09 18:04:36.
Using profile '/etc/duply/project'.
Using installed duplicity version 0.7.12, python 2.7.13, gpg 2.0.14 (Home: ~/.gnupg), awk 'GNU Awk 3.1.7', grep 'grep (GNU grep) 2.20', bash '4.1.2(2)-release (x86_64-redhat-linux-gnu)'.
Autoset found secret key of first GPG_KEY entry '78BD589D' for signing.
-- Run cmd -- Checking TEMP_DIR ‘/bup/duplicity' is a folder and writable --
test -d ‘/bup/duplicity' && test -w ‘/bup/duplicity' 2>&1
-- Run cmd -- Test - Encrypt to '78BD589D' & Sign with '78BD589D' --
echo password123 | gpg --sign --default-key 78BD589D --passphrase-fd 0 --batch -r 78BD589D --status-fd 1 -o ‘/bup/duplicity/duply.16873.1491786277_ENC' -e '/usr/bin/duply' 2>&1
-- Run cmd -- Test - Decrypt --
echo password123 | gpg --passphrase-fd 0 --batch -o ‘/bup/duplicity/duply.16873.1491786277_DEC' -d ‘/bup/duplicity/duply.16873.1491786277_ENC' 2>&1
-- Run cmd -- Test - Compare --
test "$(cat '/usr/bin/duply')" = "$(cat ‘/bup/duplicity/duply.16873.1491786277_DEC')" 2>&1
Cleanup - Delete ‘/bup/duplicity/duply.16873.1491786277_*'(FAILED)
--- Start running command STATUS at 18:04:37.283 ---
TMPDIR=‘/bup/duplicity' PASSPHRASE=password123 FTP_PASSWORD=‘PASSWORD' '/usr/local/bin/python' '/usr/local/bin/duplicity' collection-status --archive-dir ‘/bup/duplicity/.duply-cache' --name duply_project --encrypt-key 78BD589D --sign-key 78BD589D --verbosity '4' --s3-use-rrs --asynchronous-upload 's3://address@hidden/backup-project'
--- Finished state OK at 18:04:37.380 - Runtime 00:00:00.097 ---
_______________________________________________
Duplicity-talk mailing list
address@hidden <mailto:address@hidden>
https://lists.nongnu.org/mailman/listinfo/duplicity-talk
|