Re: Backups not working after running out of space

[qx6uwumzvv]

Thanks Eric,

Idea #1 didn't help (none of the .gz files where corrupt), but the 
diagnostic messages I got running idea #2 helped me realize that I 
needed to remove a(nother) current_mirror, and after I did that the 
regress succeed, and a subsequent backup also succeeded.  After that, 
however, I took your initial advice, renamed that repo (I'll keep it 
around until my daily remove increments --older-than 1Y has removed all 
it's increments) and created a new repo from scratch last night.

Thanks for your help.

On 2023-11-16 04:06, Eric Lavarde Eric-at-Lavar.de |rdiff-backup-users| 
wrote:
> Hi,
>
> it gets difficult, once a shortage in storage happens, you need to be 
> lucky for rdiff-backup to break at a place where it can recover from. 
> My recommendation would be to create a new repo from scratch, and 
> forget about the broken one. That's I would anyway because even if you 
> manage somehow to recover, you'll never know if it's absolutely clean.
>
> Few ideas though how you might be able to recover:
>
> 1. write a small script to verify all compressed files (.gz) and 
> remove them if they're corrupt
> 2. call rdiff-backup regress with --allow-duplicate-timestamps
>
> but it's hoping for the best...
>
> KR, Eric
>
> PS: that's why I recommend all my customers to monitor at least 
> storage, from everything else you can generally recover more or less 
> gracefully. Data you can't recover.
>
> On 15.11.2023 20:52, qx6uwumzvv@liamekaens.com wrote:
> > Anyone have any suggestions?
> >
> > If nothing else, is there some way I could remove appropriate files
> > from the rdiff-backup-data directory to get my backups working again?
> >
> > thanks,
> > Peter
> >
> > On 2023-11-13 13:01, Peter Canning wrote:
> > > Here's the command I've been using, with -v 6 added, followed by the 
> > > output it produced:
> > > > C:\Users\pcanning\rdiff-backup-2.2.2-64\rdiff-backup.exe -v 6 
> > > > backup --print-statistics `
> > > >             --exclude "H:/Lightroom/Processing Catalog/Backups" `
> > > >             --exclude "H:/Lightroom/Processing Catalog/Processing 
> > > > Catalog Previews.lrdata" `
> > > >             --exclude "H:/Lightroom/Processing Catalog/Processing 
> > > > Catalog-v12 Previews.lrdata" `
> > > >             "H:/Lightroom/Processing Catalog" `
> > > > \\nas4free.local\lightroom-catalogs-backups\Lightroom-Processing-Catalog.rdiff-backup 
> > >
> > > > C:\Users\pcanning\rdiff-backup-2.2.2-64\rdiff-backup.exe : WARNING: 
> > > > Previous backup seems to have failed, regressing destination now
> > > > At line:1 char:1
> > > > + C:\Users\pcanning\rdiff-backup-2.2.2-64\rdiff-backup.exe -v 6 
> > > > backup  ...
> > > > + 
> > > > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> > > >     + CategoryInfo          : NotSpecified: (WARNING: 
> > > > Previo...destination now:String) [], RemoteException
> > > >     + FullyQualifiedErrorId : NativeCommandError
> > > >
> > > > WARNING: Could not find mirror metadata file. Metadata will be read 
> > > > from filesystem instead
> > > > Fatal Error: No metadata for time Thu Nov  9 02:35:42 2023 
> > > > (1699526142) found, cannot regress
> > > > * Using repository 
> > > > '//nas4free.local/lightroom-catalogs-backups/Lightroom-Processing-Catalog.rdiff-backup'
> > > > * Hardlinks disabled by default on Windows
> > > > * Unable to import module (py)xattr. Extended attributes not 
> > > > supported on filesystem at path H:/Lightroom/Processing Catalog
> > > > * Unable to import module posix1e from pylibacl package. POSIX ACLs 
> > > > not supported on filesystem at path H:/Lightroom/Processing Catalog
> > > > * -----------------------------------------------------------------
> > > > Detected abilities for source (read only) file system:
> > > >   Access control lists                         Off
> > > >   Extended attributes                          Off
> > > >   Windows access control lists                 On
> > > >   Case sensitivity                             Off
> > > >   Escape DOS devices                           On
> > > >   Escape trailing spaces                       On
> > > >   Mac OS X style resource forks                Off
> > > >   Mac OS X Finder information                  Off
> > > > -----------------------------------------------------------------
> > > > * Directories on file system at path 
> > > > //nas4free.local/lightroom-catalogs-backups/Lightroom-Processing-Catalog.rdiff-backup/rdiff-backup-data/rdiff-backup.tmp.0 
> > > > are not fsyncable. Assuming it's unnecessary.
> > > > * Unable to import module (py)xattr. Extended attributes not 
> > > > supported on filesystem at path 
> > > > //nas4free.local/lightroom-catalogs-backups/Lightroom-Processing-Catalog.rdiff-backup/rdiff-backup-data/rdiff-ba
> > > > ckup.tmp.0
> > > > * Unable to import module posix1e from pylibacl package. POSIX ACLs 
> > > > not supported on filesystem at path 
> > > > //nas4free.local/lightroom-catalogs-backups/Lightroom-Processing-Catalog.rdiff-backup/rdiff-backup-da
> > > > ta/rdiff-backup.tmp.0
> > > > * -----------------------------------------------------------------
> > > > Detected abilities for destination (read/write) file system:
> > > >   Ownership changing                           Off
> > > >   Hard linking                                 On
> > > >   fsync() directories                          Off
> > > >   Directory inc permissions                    Off
> > > >   High-bit permissions                         On
> > > >   Symlink permissions                          Off
> > > >   Extended filenames                           On
> > > >   Windows reserved filenames                   On
> > > >   Access control lists                         Off
> > > >   Extended attributes                          Off
> > > >   Windows access control lists                 On
> > > >   Case sensitivity                             Off
> > > >   Escape DOS devices                           On
> > > >   Escape trailing spaces                       On
> > > >   Mac OS X style resource forks                Off
> > > >   Mac OS X Finder information                  Off
> > > > -----------------------------------------------------------------
> > > > * Backup: escape_dos_devices = False
> > > > * Backup: escape_trailing_spaces = False
> > > > * Enabled use_compatible_timestamps
> > > > NOTE: Symbolic links excluded by default on Windows
> > > > NOTE: Regressing to date/time Thu Nov  9 02:35:42 2023
> > > > * Cleaning up
> > >
> > >
> > > On 2023-11-13 04:57, Patrik Dufresne patrik-at-ikus-soft.com 
> > > |rdiff-backup-users| wrote:
> > > > Hello Peter,
> > > >
> > > > I apologize for the inconvenience you've experienced in recovering 
> > > > your
> > > > backup after encountering space issues. Typically, rdiff-backup can 
> > > > handle
> > > > recovery seamlessly following an increase in disk space. However, 
> > > > it seems
> > > > that the manual changes you made might have caused interference.
> > > >
> > > > To assist you more effectively, could you please share the backup 
> > > > command
> > > > line you are currently using? Additionally, I recommend executing the
> > > > command with higher verbosity by adding "-v 6." This will provide more
> > > > detailed information that can help us pinpoint the issue.
> > > >
> > > > In the expected scenario, rdiff-backup should detect the 
> > > > interruption in
> > > > the previous backup and initiate a regression of the repository to
> > > > eliminate any intermediate changes. This process is designed to 
> > > > restore the
> > > > health of your repository. Subsequently, the backup will resume its 
> > > > normal
> > > > operation.
> > > >
> > > > Your cooperation in providing the requested information will 
> > > > greatly aid us
> > > > in resolving this issue promptly. If you have any questions or 
> > > > concerns,
> > > > please don't hesitate to reach out.
> > > >
> > > > Thank you for your understanding and cooperation.
> > > >
> > > >
> > > > Le lun. 13 nov. 2023, à 00 h 12,<qx6uwumzvv@liamekaens.com>  a écrit :
> > > >
> > > > > Using rdiff-backup 2.2.2 on Windows 10 Pro 22H2, I have a 
> > > > > scheduled task
> > > > > that runs my system backup PowerShell script at 2 AM every 
> > > > > morning.  The
> > > > > script runs rdiff-backup several times to backup several different
> > > > > directories.  A few days ago one of the backups failed because the 
> > > > > NAS
> > > > > volume containing targetdir filled up.  I took some remedial steps 
> > > > > that
> > > > > didn't help, but at this point I don't remember exactly what I 
> > > > > did.  I do
> > > > > know that this first thing I tried, not thinking very careful, and
> > > > > realizing that I had recently created some large files in 
> > > > > sourcedir that I
> > > > > didn't need any more, I deleted those files and tried doing a backup
> > > > > running rdiff-backup manually (incorrectly thinking rdiff-backup 
> > > > > would
> > > > > remove some files from targetdir).  Of course since rdiff-backup is
> > > > > creating incremental backups, and targetdir was still full, that 
> > > > > backup
> > > > > failed too.  At some point I increased the size of the NAS volume, 
> > > > > and here
> > > > > is where my memory is failing me. I probably tried running 
> > > > > rdiff-backup
> > > > > manually again.   Either then, or shortly thereafter, I started 
> > > > > getting
> > > > > "regressing destination" and then rdiff-backup failing whenever I 
> > > > > tried to
> > > > > backup or verify, so I attempted to follow the instructions in FAQ 
> > > > > item
> > > > > 15.  I did some other troubleshooting that I don't remember, but the
> > > > > backups are still failing.
> > > > >
> > > > > Now, running "rdiff-backup.exe backup --print-statistics ..." doesn't
> > > > > print the expected statistics (so I assume it didn't backup 
> > > > > anything) and
> > > > > produces the following output:
> > > > >
> > > > > C:\Users\pcanning\rdiff-backup-2.2.2-64\rdiff-backup.exe : WARNING:
> > > > > Previous backup seems to have failed, regressing destination now
> > > > > At line:1 char:1
> > > > > + C:\Users\pcanning\rdiff-backup-2.2.2-64\rdiff-backup.exe backup 
> > > > > --pri ...
> > > > > + 
> > > > > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> > > > >      + CategoryInfo          : NotSpecified: (WARNING: 
> > > > > Previo...destination
> > > > > now:String) [], RemoteException
> > > > >      + FullyQualifiedErrorId : NativeCommandError
> > > > >
> > > > > WARNING: Could not find mirror metadata file. Metadata will be 
> > > > > read from
> > > > > filesystem instead
> > > > > Fatal Error: No metadata for time Thu Nov  9 02:35:42 2023 
> > > > > (1699526142)
> > > > > found, cannot regress
> > > > > NOTE: Symbolic links excluded by default on Windows
> > > > > NOTE: Regressing to date/time Thu Nov  9 02:35:42 2023
> > > > >
> > > > > Running "rdiff-backup.exe verify ..." produces the following output:
> > > > >
> > > > > C:\Users\pcanning\rdiff-backup-2.2.2-64\rdiff-backup.exe : 
> > > > > WARNING: Two
> > > > > different times for current mirror found
> > > > > At line:1 char:1
> > > > > + C:\Users\pcanning\rdiff-backup-2.2.2-64\rdiff-backup.exe verify 
> > > > > \\nas ...
> > > > > + 
> > > > > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> > > > >      + CategoryInfo          : NotSpecified: (WARNING: Two di...nt 
> > > > > mirror
> > > > > found:String) [], RemoteException
> > > > >      + FullyQualifiedErrorId : NativeCommandError
> > > > >
> > > > > WARNING: Could not find mirror metadata file. Metadata will be 
> > > > > read from
> > > > > filesystem instead
> > > > > WARNING: Mirror metadata not found, reading from directory
> > > > >
> > > > > followed by many lines saying
> > > > >
> > > > > WARNING: Cannot find SHA1 digest for file ..., perhaps because this
> > > > > feature was added in v1.1.1
> > > > >
> > > > > and ending with the line:
> > > > > NOTE: Verification found 1428 files without hash, all others could be
> > > > > verified successfully
> > > > >
> > > > > Any suggestions for how to get my backups (and verifies) working 
> > > > > again?
> > > > >
> > > > > thanks,
> > > > > Peter