Possible Tar Bug

[Chuck Wolber]

If this is the wrong place to be posting this, please sincerely excuse the
intrusion and if it is not too much trouble, please point me in the right
direction.


Background:

We do nightly full backups. We back up approximately eight filesystems
totaling approximately 30Gb. Our method is to rsync the various Linux/ext2
based filesystems to a central file server and then use tar to stream that
data to tape (DDS3 tape on HP C1554C drives). Since we have 3 DDS3 drives
on our backup server, we use a script with some logic to determine how
best to fit each filesystem to which tape. This system ensures that files
are not changing during a backup and allows us to have the previous nights
backed up files online and available for restoration (99% of restores are
from accidental deletion, hence I significantly reduced my need to restore
from tape).

Keep in mind that we do *NOT* stream 30GB to tape all at once. We have
approximately eight filesystems we rsync and then we run those to tape one
at a time. The largest filesystem is about 9.5GB and the smallest is about
1GB. Each contains anything that would be found from "/" onward on a
typical RedHat 7.x installation. The problem described below manifests 
itself on all of the filesystems we back up.


The Problem:

When including the --verify flag with the tar command one of two scenarios
occur.

1) The complete archive is written to tape and then after about 20% of the
verify has completed the following message is sent to STDERR and then tar
exits:

/bin/tar: memory exhausted

I am not clear on what memory has been exhausted. There is plenty of
unused memory available at the time. I suspect that this may be some sort
of internal tar limitation, though I have not yet gone through the source
code to confirm this.


2) The complete archive is written to tape and then tar hangs after
approximately 20% of the verify has completed.


I started seeing the "/bin/tar: memory exhausted" once I upgraded to the
tar version distributed with RedHat 7.2 (tar-1.13.19-6), although on
extremely large filesystems (like our 9.5GB one) it will hang as well. The
versions that would hang consistently were the ones that came with the
RedHat 7.0 and 7.1 distributions (tar-1.13.17-8, tar-1.13.19-4). None of
them left any sort of information in any of the log files.


-- 
Chuck Wolber
System Administrator
AltaServ Corporation
(425)576-1202
address@hidden    "Condense facts from the vapor of nuance."