[Lzip-bug] Tarlz 0.5 released

[Antonio Diaz Diaz]

I am pleased to announce the release of tarlz 0.5.

I dedicate with admiration this release to the 2 million people that the 
1 of October of 2017 defended with their bodies the ballot-boxes of the 
catalan independence referendum. Spanish government should have 
negotiated with that of Catalonia instead of sending the police to bash 
the voters. Why some governments still think they have the right to 
mistreat their citizens?

Tarlz is a small and simple implementation of the tar archiver. By 
default tarlz creates, lists and extracts archives in a simplified posix 
pax format compressed with lzip on a per file basis. Tarlz can append 
files to the end of such compressed archives.

Each tar member is compressed in its own lzip member, as well as the 
end-of-file blocks. This same method works for any tar format (gnu, 
ustar, posix) and is fully backward compatible with standard tar tools 
like GNU tar, which treat the resulting multimember tar.lz archive like 
any other tar.lz archive.

Tarlz can create tar archives with four levels of compression 
granularity; per file, per directory, appendable solid, and solid.

Of course, compressing each file (or each directory) individually is 
less efficient than compressing the whole tar archive, but it has the 
following advantages:

   * The resulting multimember tar.lz archive can be decompressed in
     parallel with plzip, multiplying the decompression speed.

   * New members can be appended to the archive (by removing the eof
     member) just like to an uncompressed tar archive.

   * It is a safe posix-style backup format. In case of corruption,
     tarlz can extract all the undamaged members from the tar.lz
     archive, skipping over the damaged members, just like the standard
     (uncompressed) tar. Moreover, lziprecover can be used to recover at
     least part of the contents of the damaged members.

   * A multimember tar.lz archive is usually smaller than the
     corresponding solidly compressed tar.gz archive, except when
     individually compressing files smaller than about 32 KiB.

Note that the posix pax format has a serious flaw. The metadata stored 
in pax extended records are not protected by any kind of check sequence. 
Because of this, tarlz implements a CRC of the extended records, and the 
option '--ignore-crc' to ignore a missing CRC in archives not created by 
tarlz.

The diagram below shows the correspondence between each tar member 
(formed by one or two headers plus optional data) in the tar archive and 
each lzip member in the resulting multimember tar.lz archive:

tar
+========+======+=================+===============+========+======+========+
| header | data | extended header | extended data | header | data |   eof  |
+========+======+=================+===============+========+======+========+

tar.lz
+===============+=================================================+========+
|     member    |                      member                     | member |
+===============+=================================================+========+

Tarlz is intended as a showcase project for the maintainers of real tar 
programs to evaluate the format and perhaps implement it in their tools.


The homepage is at http://www.nongnu.org/lzip/tarlz.html

An online manual for tarlz can be found at 
http://www.nongnu.org/lzip/manual/tarlz_manual.html

The sources can be downloaded from
http://download.savannah.gnu.org/releases/lzip/tarlz/tarlz-0.5.tar.lz

The sha256sum is:
d7e7e4bda401f43537d2805101223db572aab0b2f9081a1c88fe350231c5a1a0 
tarlz-0.5.tar.lz


Changes in version 0.5:

  * A simplified posix pax format has been implemented. Tarlz can now 
store, list and extract:
  - files with unlimited name size
  - links to files with unlimited name size
  - files of unlimited size

  * A CRC32-C (Castagnoli) of the extended header data has been 
implemented. This fixes a serious flaw in the posix pax format. A CRC 
was chosen because a checksum is too weak for a potentially large list 
of variable sized records. A checksum can't detect simple errors like 
the swapping of two bytes.

  * The new option '--ignore-crc' has been added.

  * Missing #includes for major, minor and makedev have been added.

  * The new archive format has been documented in the manual.


Please send bug reports and suggestions to address@hidden


Regards,
Antonio Diaz, tarlz author and maintainer.

--
If you are distributing software in xz format, please consider using 
lzip instead. See http://www.nongnu.org/lzip/lzip_benchmark.html#xz1 and 
http://www.nongnu.org/lzip/xz_inadequate.html