Re: [Duplicity-talk] .Cache & duplicity collection-status

[Сергей Цаболов]

Hello,

12.07.2021 16:29, Scott Hannahs via Duplicity-talk пишет:

>       
>       Basically your disk is so full that duplicity can not find the
>       space for the files to keep track of the backups.  These files
>       will continue to grow as you add more and more incremental
>       backups. It looks like you are storing the backups on a mounted
>       server or locally in /srv?

I don't understand something.

 I can  take incremental backups every day?  I think yes.

The full system I take once a week, after some time old backups deleted in full and incremental/

    /srv is the NFS share from TrueNas Server

If you are running your system with the root dir over 90% full you are in for some performance issues anyway.

Not, is not over 90% something over 68-70% only.

Now with Symlnik from /root/.cache/duplicity -> /srv/duplicity/.cache/duplicity is worked fine, test command worked fine.

With commands  :

duplicity full --no-encryption --volsize=800 --exclude=/srv --exclude=/run --exclude=/proc --exclude=/sys --exclude=/dev --exclude=/mnt --exclude=/media --exclude=/tmp --exclude=/var/spool --exclude=/var/cache --exclude=/var/tmp --exclude=/swap.img / file:///srv/duplicity/

duplicity incremental --no-encryption --volsize=800 --exclude=/srv --exclude=/run --exclude=/proc --exclude=/sys --exclude=/dev --exclude=/mnt --exclude=/media --exclude=/tmp --exclude=/var/spool --exclude=/var/cache --exclude=/var/tmp --exclude=/swap.img / file:///srv/duplicity

some issues is they will leave.

You should probably focus on backing up files that are necessary rather than brute force trying to backup the whole root partition and OS that can easily be reloaded.

The concept is backing up full system,  duplicity is very good and flexible for me , very good and easy restore some folder or files if need.

-Scott

On Jul 12, 2021, at 08:53, Сергей Цаболов via Duplicity-talk <duplicity-talk@nongnu.org> wrote:

Hello,

12.07.2021 14:38, edgar.soldin--- via Duplicity-talk пишет:

On 12.07.2021 13:31, Сергей Цаболов wrote:

Hello,

12.07.2021 13:31, edgar.soldin--- via Duplicity-talk пишет:

On 12.07.2021 10:33, Сергей Цаболов via Duplicity-talk wrote:

Hello to all.

I have one question.

I setup duplicity and is work very well .

I setup the cron job for backup  incremental with command :

duplicity incremental --no-encryption  --exclude=/run --exclude=/proc --exclude=/sys --exclude=/dev --exclude=/proc --exclude=/sys --exclude=/mnt --exclude=/media --exclude=/tmp --exclude=/var/spool --exclude=/var/cache --exclude=/var/tmp --exclude=/swap.img / file:///srv/duplicity/

did you notice that you doubled some exclusions above?

Yes, I  notice my doubled exclusions I think is not so serious typo

I run is by root user.

In /root/.cache/duplicity  is collect some files very big and my root-file-system (/) is full 100%

that's the archive dir and needed for duplicity to work. man page [1] says
"
To save bandwidth, duplicity generates full signature sets and incremental signature sets. A full signature set is generated for each full backup, and an incremental one for each incremental backup. These start with duplicity-full-signatures and duplicity-new-signatures respectively. These signatures will be stored both locally and remotely. The remote signatures will be encrypted if encryption is enabled. The local signatures will not be encrypted and stored in the archive dir (see --archive-dir ).

In the man [1] page I found that paragraph and now understand why is placed the files there

"

be aware that the mentioned argument above allows you to place the archive dir anywhere in case you run into space issues like you did.

I found :

The interaction between the --archive-dir and the --name options allows for four possible combinations for the location of the archive dir:

              1.     neither specified (default)
                      ~/.cache/duplicity/hash-of-url

              2.     --archive-dir=/arch, no --name
                      /arch/hash-of-url

              3.     no --archive-dir, --name=foo
                      ~/.cache/duplicity/foo

              4.     --archive-dir=/arch, --name=foo
                      /arch/foo

Which one is suitable for my case , for like now I moved the .cache/duplicity in the storage and make Symlink to backup the /root/.cache/when backup moved over nfs.

just point --archive-dir to wherever is enough space! e.g. --archive-dir=/mnt/bigspace/.duplicity-cache

1. what are the sizes of your partitions? maybe provide the output of 'df -h'

Is one partition LVM 45G

the root partition?

Yes, is the root partition

2. what's the size of your backup, how many files?

I backup all files of the root file system path /  (with some exclude)

did you exclude the archive-dir from your backup? not sure i've seen it in your excludes.

Yes, in cron job i exclude it.

Before I delete it, the command  duplicity collection-status  file:///srv/duplicity/ give Num volumes  duplicity-inc .

After I delete the biggest  files from .cache the info about  Num volumes  duplicity-inc not show again only  the Full Num volumes

you are not supposed to manually edit that folder, although duplicity should survive and recreate missing data on the next run

Ok,  right now I understand it, when I delete it :  duplicity collection-status not show inc backup, but files stay in the place when all backup.

How I can change the backup command to not collect sow big .cache files, because if I run the duplicity cleanup --extra-clean --force $target  the info about Num volumes of full and inc backups I will have ?

you can't change the size per se. if you have really big files to back up consider raising max-blocksize to lower the size of your sigtar files

No, I not have many big files to backup, I backup full system, I understand I can to include what I need to backup and situation changed (but is not so simple in my case).
I forget to write, files for inc grated  in cache for 2 days with cron job.

sorry. what do you mean by "files for inc grated  in cache for 2 days with cron job."?

I mean: cron job for duplicity incremental run every day

10 21 * * * duplicity incremental --no-encryption  --exclude=/root/.cache/duplicity --exclude=/run --exclude=/dev --exclude=/proc --exclude=/sys --exclude=/mnt --exclude=/media --exclude=/tmp --exclude=/var/spool --exclude=/var/cache --exclude=/var/tmp --exclude=/swap.img / file:///srv/duplicity/

Available space in disk take once in 2021/07/10,  next day not enough space for cached files because disk is full 100%.

And second cron job once in the week can't finished :

30 4 * * sun duplicity full --no-encryption  --exclude=/run --exclude=/srv --exclude=/dev --exclude=/proc --exclude=/sys --exclude=/mnt --exclude=/media --exclude=/tmp --exclude=/var/spool --exclude=/var/cache --exclude=/var/tmp --exclude=/swap.img / file:///srv/duplicity/duplicity

_______________________________________________
Duplicity-talk mailing list
Duplicity-talk@nongnu.org
https://lists.nongnu.org/mailman/listinfo/duplicity-talk

-- 
-------------------------
Sergey Tsabolov