duplicity-talk
[Top][All Lists]
Advanced
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Duplicity-talk] Failures on backup jobs from cron

From: Russell Clemings
Subject: Re: [Duplicity-talk] Failures on backup jobs from cron
Date: Tue, 7 Mar 2023 14:03:58 -0800
Answers below.

On Tue, Mar 7, 2023 at 7:32 AM Kenneth Loafman <kenneth@loafman.com> wrote:
Hi Russel,

1) Which version(s) of duplicity, Python, OS are you using?  Which work and which don't.

Duplicity 1.2.2, Python 3.8.3, CentOS v7.9.2009 STANDARD virtuozzo with cPanel 106.0.18
 

2) As to your problem with the cryptography module install, try:

    $ python3 -m pip install --U cryptography::38.0.4

    or ignore it completely if you are not using azure backend.

I don't have azure but tried it anyway: ERROR: Invalid requirement: 'cryptography::38.0.4'

(Should "--U" be "-U"? I got "unknown option" with "--U".)
 

3) On the cron job, make sure all paths are set properly in crontab, and make sure you don't have duplicate duplicity instances:

    $ find /usr -name duplicity -type f
    $ find /usr -name duplicity -type d

I do get multiple versions, but I've always included the full path in my script, e.g.:

crontab: /home/newton/duplicity-0.8.12.1612/s3backup/backup.sh
backup.sh: /home/newton/duplicity-0.8.12.1612/bin/duplicity
 
    If you have more than one response from above, uninstall all, and reinstall the one you want.

I'm kind of afraid to uninstall anything until I can get a replacement working. Right now 0.8.12.1612 still works from the command line.

4) If you installed from source, just remember that the source is non-versioned.  We use the last tag as the version during release.
    There are multiple options both stable and latest:

I did install from source. Maybe that's the problem. 

Before I gave up on fixing the 0.8.12.1612 installation (which as I said still works from the command line, just not from cron), I was getting this error when running from the root cron:

ImportError: cannot import name '_librsync'

So I'm stuck. Unless there are other suggestions I guess I will try one of the options below and hope for the best. What's bewildering is everything had been working fine for several years, until it started failing on two of four servers a couple of weeks ago.

    Since the snap versions have all the pypi stuff resolved that may be your best bet.

I hope some of this helps.

...Thanks,
...Ken




On Mon, Mar 6, 2023 at 4:19 PM Russell Clemings via Duplicity-talk <duplicity-talk@nongnu.org> wrote:
OK, well, I gave up. Everytime I tracked down and included a dependency (via PYTHONPATH), I'd just get another failure from another missing dependency.

So I decided that if I was going to spend that much time on it, I might as well just start fresh with the current version, following the instructions here:


It seems to have installed OK, except for an error with pip3 install -r requirements.txt:

---

Collecting cryptography>=2.1.4 (from azure-storage-blob->-r requirements.txt (line 18))
  Using cached https://files.pythonhosted.org/packages/fa/f3/f4b8c175ea9a1de650b0085858059050b7953a93d66c97ed89b93b232996/cryptography-39.0.2.tar.gz
    Complete output from command python setup.py egg_info:

            =============================DEBUG ASSISTANCE==========================
            If you are seeing an error here please try the following to
            successfully install cryptography:

            Upgrade to the latest pip and try again. This will fix errors for most
            users. See: https://pip.pypa.io/en/stable/installing/#upgrading-pip
            =============================DEBUG ASSISTANCE==========================

    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-build-g7v1tzdv/cryptography/setup.py", line 18, in <module>
        from setuptools_rust import RustExtension
    ModuleNotFoundError: No module named 'setuptools_rust'

    ----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-g7v1tzdv/cryptography/

---

Upgrading pip as suggested didn't do anything:

---

xxxxxx@xxxx2 [~/duplicity-main]# python3 -m pip install --upgrade pip
Defaulting to user installation because normal site-packages is not writeable
Requirement already satisfied: pip in /usr/local/python3.8/lib/python3.8/site-packages (23.0.1)

---

Finally when I try to run a backup script, I get what I think are some boto errors:

---

Attempt of list Nr. 1 failed. ClientError: An error occurred (403) when calling the HeadBucket operation: Forbidden
Attempt of list Nr. 2 failed. AttributeError: 'NoneType' object has no attribute 'objects'
Attempt of list Nr. 3 failed. AttributeError: 'NoneType' object has no attribute 'objects'
Attempt of list Nr. 4 failed. AttributeError: 'NoneType' object has no attribute 'objects'
Giving up after 5 attempts. AttributeError: 'NoneType' object has no attribute 'objects'

---

Suggestions?





On Mon, Feb 27, 2023 at 2:51 PM Russell Clemings <rclemings@gmail.com> wrote:
Thanks. I'll try these and report back.

On Mon, Feb 27, 2023 at 2:03 PM edgar.soldin--- via Duplicity-talk <duplicity-talk@nongnu.org> wrote:
such issues usually are caused by cron not setting PATH env var  or wrongly. in my experience ,)

to debug it, run 'which python3'
1. in cron. pipe to file or logger to retrieve the output
2. as root
comparing both ourputs you will probably see that different python interpreters are used, with one not having the missing modules installed in it's default location

resolve by adding the PATH env var of the root shell at crontab top.

good luck.. ede/duply.net




On February 27, 2023 9:48:24 PM GMT+01:00, Scott Hannahs via Duplicity-talk <duplicity-talk@nongnu.org> wrote:
You may need to set the PYTHONPATH variable when running from Cron or other non-shell systems.

-Scott

On Feb 27, 2023, at 14:37, Russell Clemings via Duplicity-talk <duplicity-talk@nongnu.org> wrote:

I've got a curious problem suddenly with duplicity on two of more four servers.

If I run a backup job as root, everything works fine. If it runs from the root cron, I get this:

First server:
Traceback (most recent call last):
  File "/home/xxxx/duplicity-0.8.12.1612/bin/duplicity", line 30, in <module>
    from future import standard_library
ModuleNotFoundError: No module named 'future'

Second server:
Traceback (most recent call last):
  File "/home/xxxx/duplicity-0.8.12.1612/bin/duplicity", line 39, in <module>
    import fasteners
ModuleNotFoundError: No module named 'fasteners'

Both modules are installed, and as I said, the backups run fine from the command line. Also, it works fine from cron on two other servers. (And I realize my duplicity is a few versions behind; updating is on my to-do list.)

Any pointers to help locate the problem are appreciated.

--
===============================================
Russell Clemings
<russell@clemings.com>
===============================================
_______________________________________________
Duplicity-talk mailing list
Duplicity-talk@nongnu.org
https://lists.nongnu.org/mailman/listinfo/duplicity-talk

_______________________________________________
Duplicity-talk mailing list
Duplicity-talk@nongnu.org
https://lists.nongnu.org/mailman/listinfo/duplicity-talk


--
===============================================
Russell Clemings
<russell@clemings.com>
===============================================


--
===============================================
Russell Clemings
<russell@clemings.com>
===============================================
_______________________________________________
Duplicity-talk mailing list
Duplicity-talk@nongnu.org
https://lists.nongnu.org/mailman/listinfo/duplicity-talk


--
===============================================
Russell Clemings
<russell@clemings.com>
===============================================
reply via email to
[Prev in Thread] Current Thread [Next in Thread]