Re: [Mingw-cross-env-list] Cleaning individual packages before build

[Volker Grabsch]

Tony Theodore schrieb:
> From time to time I get bitten by previous versions of a package
> interfering with an updated version. The standard solution is "make
> clean", but sometimes it isn't apparent that there is a problem until
> later.

I'm experiencing this problem from time to time, too. And this
whole proposal has already been brought up some time ago. My
opinion didn't change since then:

Yes, some kind of versioning/uninstall mechanism would be nice.
However, this is a highly nontrivial task, so it should be done
using mature tools that were designed for that task. An ad-hoc
solution using some shell commands in the main Makefile will
definitely do more harm than good. [1]

That's why I rejected that proposal in the past, and why I'm
rejecting it now - unless, of course, the vital issues are
addressed.

Also note that the "update issue" doesn't happen very often.
so if the solution isn't rock-stable, but fails from time
to time, we didn't accomplish anything.

> I've attempted some simple modifications [1] to the Makefile that
> diffs the directory structure and deletes anything previously
> installed by each package. It doesn't introduce much overhead, .5
> seconds for the last package (zziplib) when usr/ is at it's fullest.
> 
> Is this a good approach, and does anyone see this as a problem?

In general, I see the following issues:

1) A diff of file listings only sees added and removed files,
   but no changed files. Yes, this doesn't happen often, but
   still needs to be addressed.

2) As a minor issue, a removed file will only be noticed when
   it's too late, i.e. after it has been removed. This might
   not be a big issue since build scripts aren't expected to
   remove system files. Nevertheless, this should be detected
   and handled.

3) The approach of taking a snapshot before and after the
   build won't work when inter-package parallelization is
   enabled (e.g. "make -j 4"). It only works if the packages
   are built one after another. Otherwise, it will do horribly
   wrong.

4) Nobody yet found a simple but reliably working solution to
   this problem. Maybe because there just isn't a simple one.
   Hoewever, lots of reliable tools have been developed that
   may be of great help for us:

       a) package systems  (like dpkg and rpm)
       b) version control systems  (like Mercurial and Git)
       c) rsnapshot  (i.e. hard links + rsync)

   Integrating one of those seems to be the only reliable
   way. Also, even without an integration, these tools
   can be used outside of mingw-cross-env.

5) The solution should be well-tested. In particular, there
   should be at least one person who uses that tool manually
   (or maybe already integrated it into mingw-cross-env) and
   can confirm that doesn't raise any unexpected issues when
   using it for a month or so.

I believe there were some more issues, but I think those
five cover already the most important aspects.


Greets,
Volker




[1]

As a side note, in the proposed patch the following line
caught my eye. There are so many things wrong with it ...
it's hard to believe that it even works. But it does, if
only by accident, with lots of error messages to be expected.
The suppression of those error messages at the end of the
statement finally tops it all:

    time rm -rfv `$(SED) 's/^> \(.*\)/\1/' '$(PREFIX)/installed/$(1)-files'` 
2>/dev/null

I really hope you don't take this criticism personally, but
independently from the current discussion, I don't ever
want to see that kind of code on a production system. :-)

-- 
Volker Grabsch
---<<(())>>---