Re: [PATCH] maint: RFC: add lzip tarballs

[Assaf Gordon]

Hello Antonio,

Thank you for writing and explaining some of these issues.

However, my personal opinion is that these explanations are not helping in 
convincing that lzip is ready for production use. In fact, some of them raise 
more "red flag" for me.

To elaborate:

> On Jan 11, 2017, at 21:00, Antonio Diaz Diaz <address@hidden> wrote:
> 
> Assaf Gordon wrote:
>> I do have some concerns about the lzip program:
>> 
>> It is written in C++. Not a problem by itself, but seems a bit at odds as a 
>> requirement for system-level package like coreutils.
> 
> I maintain 3 lzip (de)compressors written in C:

I think only makes things look worse... developer's time is limited - why 
maintain so many overlapping projects separately ?

Especially when trying to convince people to use this project - which one 
should they choose?

While there's 'gzip' vs 'gunzip', 'bzip2' vs 'bunzip2' and 'xz' vs 'unxz' - 
here you present lzip,plzip,pdlzip,lunzip,clzip - not only confusing, but also 
annoying if this needs to be used in a script:
If my script calls 'lzip' - I force the user to install the 'c++/GPL' version, 
regardless of whether they want or can install.

And the versions of them are not the same: 
  pdlzip is at 1.7
  plzip  is at 1.5 (and 1.6pre1)
  lunzip and lzlib are at 1.8
  lzip is at 1.19pre2

Are they out-of-sync, or simply some parts did not need updating?
As a potential user - I have no easy way to know if I'm up to date or not.
Not only that - if there is ever an update, lots of manual work is required to 
install all the updated packages.

If you want to gain traction, I would suggest consolidating all into one simple 
package (one tarball, one version) for all the critical executables.


>> I'm not sure how portable and well-tested the program is on the large number 
>> of platforms that coreutils aim to cater to.
> 
> Lzip itself is pretty portable. The C versions are even more portable (clzip 
> has been compiled wit gcc 2.7.2 IIRC). Lunzip provides a "low memory" mode 
> able to decompress any file using as little memory as 50 kB, irrespective of 
> the dictionary size used to compress the file. It has been tested on a laptop 
> with a 486 processor and 4 MiB of RAM.
> 
> About how well-tested lzip is, you can see 
> http://www.nongnu.org/lzip/manual/lzip_manual.html#Quality-assurance

> The test suite run by 'make check' is just a minuscule subset of the full 
> tests.

Sorry, but I don't consider  486 "portability testing"  nor statements in a web 
page as "quality assurance".

I do not doubt you've tested things extensively by yourself, but it simply does 
not seem so from the code.

For starters, I would expect a "make coverage" to automatically run tests and 
generate coverage information.
Without it - it seems like you'll always be doing it manually, and that's a 
sure way to miss something.

Then, once you do have automated coverage - add enough tests to actually cover 
the majority of your code.
Form a very cursory look, it seems the test suite covers about %80 of the lines 
and %60 of the branches.
Perhaps you've tested it manually, perhaps you have a superb internal coverage 
and test suite - but it simply doesn't show - and that does not encourage 
confidence in lzip.

I would suggest to add coverage tests, then add sufficient tests, then consider 
joining the gcc compile farm and test it on all of their systems.
There are other gnu members who are experts in testing on many different 
platforms (I'll let them chime in if they wish) - and once you have good tests, 
it's easy for them to help you test it.

---

Then there's the development model:
Eric already mentioned that other projects have also been somewhat closed in 
their development.
That certainly happens. But I think that a free-software project that
wants to become popular in this day and age must be public in its development.
There is no excuse today to keep things closed and only release tarballs when
you feel like it. You are trying to enter an already crowded market
(gzip,bzip2,xz, then snappy, brotli and zstandard and others), and the burden
of convincing the community lzip is production ready is on you.

Making the development process public will help potential users and developers
to see how code is improved, how bugs are fixed, and how contributions and 
handled.



Lastly, at the risk of starting a flame war,
I'll mention the license:
I humbly think that a new system-level project that aims to be adopted 
universally
will have less chance to succeed today while being GPL-version-X-or-later.
(there are many existing universal projects which are GPL-X-or-later - but when 
they started -
they provided new functionality - here i'm talking about a new project).

before readers start sending me hate mail,
I want to emphasize I'm not against GPL.
Obviously I'm member of coreutils and sed - both are GPLv3-or-later,
my own project GNU Datamash is GPLv3-or-later,
and I think GPL is fantastic.

But the reality is that there are other systems and environments 
where GPLv-X-or-later is frowned upon, and they actively avoid it.
'git' is an example of a recent big universal project that was GPL,
and the fact that it was GPLv2-only greatly helped - it ensured
that it won't ever become GPLv3.

You've already downgraded your license form GPLv3 to GPLv2-or-later - 
so you are aware of these issues.

You are also providing a "Public Domain" C version - but it has limited
functionality - so that's not encouraging to use it.

If 'lzip' was a novel project that provided some very new and very desired
functionality then GPL license would be less of a barrier for adoption (and 
in fact, I'm sure that's exactly what GNU/FSF want - fantastic projects under
GPL that would help adoption, and I fully support that).

But realistically, 'lzip' is an improvement of an existing functionality,
not a novel one. Having "GPL-X-or-later" license will not be an obstacle to any
GNU/Linux system, but will be a problem for some other systems or communities
(in regards to being natively available on the system. Clearly users can
always install it manually).

So if wide universal adoption is a goal - the license might become an issue (at 
the very least for the library part of lzlib).



regards,
 - assaf