Re: [Duplicity-talk] Restoring a file deleted before the latest incremental

[Nate Eldredge]

On Mon, 21 Aug 2023, duplicity--- via Duplicity-talk wrote:

> Hi all, I am executing a somewhat complex test plan which I will share if you 
> need it to give advice, but here is the short version first. I generally leave 
> my partitions mounted and have a partition by partition backup that runs 
> nightly.
>
> What happens if the following timeline occurs?
>
> 1) Full backup, some number of incrementals, each with a given file changed.
> 2) LUKS partition is inadvertently dismounted or system boots w/o me manually 
> mounting partitions.
> 3) Scheduled backup occurs on raw mount point.
> 4) LUKS partition is mounted on raw mount point.
> 5) Changed file is deleted.
> 6) Next scheduled backup is run.

What I would expect is that the #3 backup is empty, as if all the files 
had been deleted, and that #6 behaves as if all the files had been newly 
created again.  (Unless there is something in the mount point directory 
that gets hidden when the partition is mounted.)

> At this point the changed file from #1 is no longer in the catalogue 
> because of #3. I have tried restoring by timestamp previous to #2, but 
> that older file isn't available. I see two options:

That shouldn't happen.  In my tests, using --time and --path-to-restore, 
it works fine to restore a file that existed at some previous time, even 
if that file does not exist in the most recent backup.

Are you certain that you specified a timestamp between #1 and #3? 
Sometimes time zones can lead to mistakes.

Can you reproduce this with a smaller test case, showing the exact 
commands you ran to do the backups and restores?

> 1) Run a backup when I mount the partition in #4, but since some of my 
> partitions are large and my jobs run 4-7 hours I would like to find 
> another way. Also, #5 could happen prior to that backup completing.

Having occasional backups where the directory is empty shouldn't hurt 
their integrity, but is inefficient if you are doing incrementals.  In 
your example, if backup #6 is an incremental, it will have to contain a 
complete copy of all the files, since in the previous backup #3, none of 
them existed.  If backup #3 had not taken place, then #6 would only have 
to contain the diffs since #1.

One idea is to have your automated backup job test that the partition is 
mounted before doing the backup.  A simple way is to choose a file that 
should always be present in the mounted partition (or create a dummy file 
for this purpose), and test that it exists.  E.g. a cron job like

0 0 * * * [ -e /mnt/foo/myfile ] && duplicity ...

> 2) I found the following post, but don't understand "... If you had the
> exact time of deletion and moved any incremental backups out of the way
> that had occurred since then, it would be possible to do it manually."
> https://lists.gnu.org/archive/html/duplicity-talk/2008-05/msg00018.html

I don't think that post is relevant to your use case.  There, the 
user wanted to recover a file *without* knowing the timestamp when it 
existed, which isn't supported.  In your case, it sounds like you do know 
the timestamp.

--
Nate Eldredge
nate@thatsmathematics.com