[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Lzip-bug] Tarlz 0.21 released

From: Antonio Diaz Diaz
Subject: [Lzip-bug] Tarlz 0.21 released
Date: Thu, 17 Jun 2021 17:28:43 +0200
User-agent: Mozilla/5.0 (X11; U; Linux i586; en-US; rv: Gecko/20110420 SeaMonkey/2.0.14

I am pleased to announce the release of tarlz 0.21.

Tarlz is a massively parallel (multi-threaded) combined implementation of the tar archiver and the lzip compressor. Tarlz uses the compression library lzlib.

Tarlz creates tar archives using a simplified and safer variant of the POSIX pax format compressed in lzip format, keeping the alignment between tar members and lzip members. The resulting multimember tar.lz archive is fully backward compatible with standard tar tools like GNU tar, which treat it like any other tar.lz archive. Tarlz can append files to the end of such compressed archives.

Keeping the alignment between tar members and lzip members has two advantages. It adds an indexed lzip layer on top of the tar archive, making it possible to decode the archive safely in parallel. It also minimizes the amount of data lost in case of corruption. Compressing a tar archive with plzip may even double the amount of files lost for each lzip member damaged because it does not keep the members aligned.

Tarlz can create tar archives with five levels of compression granularity: per file (--no-solid), per block (--bsolid, default), per directory (--dsolid), appendable solid (--asolid), and solid (--solid). It can also create uncompressed tar archives.

Of course, compressing each file (or each directory) individually can't achieve a compression ratio as high as compressing solidly the whole tar archive, but it has the following advantages:

    * The resulting multimember tar.lz archive can be decompressed in
      parallel, multiplying the decompression speed.

    * New members can be appended to the archive (by removing the EOF
      member), and unwanted members can be deleted from the archive.
      Just like an uncompressed tar archive.

    * It is a safe POSIX-style backup format. In case of corruption,
      tarlz can extract all the undamaged members from the tar.lz
      archive, skipping over the damaged members, just like the standard
      (uncompressed) tar. Moreover, the option '--keep-damaged' can be
      used to recover as much data as possible from each damaged member,
      and lziprecover can be used to recover some of the damaged members.

    * A multimember tar.lz archive is usually smaller than the
      corresponding solidly compressed tar.gz archive, except when
      individually compressing files smaller than about 32 KiB.

Note that the POSIX pax format has a serious flaw. The metadata stored in pax extended records are not protected by any kind of check sequence. Because of this, tarlz protects the extended records with a Cyclic Redundancy Check (CRC) in a way compatible with standard tar tools.

The homepage is at http://www.nongnu.org/lzip/tarlz.html

An online manual for tarlz can be found at http://www.nongnu.org/lzip/manual/tarlz_manual.html

The sources can be downloaded from

The sha256sum is:
0f972112dd3f126a394d5a04107695ccbbb9e603e7b193367bd1714607e0adf4 tarlz-0.21.tar.lz

Changes in version 0.21:

  * Lzlib 1.12 or newer is now required to compile and run tarlz.

* Members without name are now skipped when decoding except when listing. This allows the reliable detection of certain format violations during parallel extraction with more than two threads. (Thanks to Florian Schmaus for reporting the problem).

* The new option '-z, --compress', which compresses existing POSIX tar archives aligning the lzip members to the tar members with choice of granularity, has been added. Existing compressed archives are not overwritten.

* The new option '-o, --output', which writes the compressed output to a file, has been added. Currently '--output' only works with '--compress'.

* The new option '--warn-newer', which warns during archive creation if any file being archived has a modification time newer than the archive creation time, has been added.

  * A failure in the '--diff' test of the testsuite on OS/2 has been fixed.

Please send bug reports and suggestions to lzip-bug@nongnu.org

Antonio Diaz, tarlz author and maintainer.

If you care about data safety and long-term archiving, please consider using lzip. See http://www.nongnu.org/lzip/lzip_benchmark.html
http://www.nongnu.org/lzip/manual/lzip_manual.html#Quality-assurance and
http://www.nongnu.org/lzip/safety_of_the_lzip_format.html Thanks.

reply via email to

[Prev in Thread] Current Thread [Next in Thread]