Re: [lmi] [PATCH] Allow installing wxWidgets from GitHub

[Greg Chicares]

On 2015-08-05 11:43, Vadim Zeitlin wrote:
[...]
>  For now, no patches but just one question about something that puzzled me
> a lot: why do we use --timestamping option with wget when we only invoke it
> when the file doesn't exist?

For example, 'install_wx.make' says this (similar to other makefiles):

.PHONY: %.tar.bz2 %.tar.gz
%.tar.bz2 %.tar.gz:
        cd $(cache_dir) && [ -e $@ ] || $(WGET) $(WGETFLAGS) $(address@hidden)
        cd $(cache_dir) && $(ECHO) "$(address@hidden) *$@" | $(MD5SUM) --check
        -$(TAR) --extract $(TARFLAGS) --directory=$(wx_dir) 
--file=$(cache_dir)/$@

For wx-3.0.0, the variables of interest would be:

$@         wxWidgets-3.0.0.tar.bz2
$(address@hidden)  ftp://ftp.wxwidgets.org/pub/3.0.0/wxWidgets-3.0.0.tar.bz2
$(address@hidden)  241998efc12205172ed24c18788ea2cd

Thus: test whether we have already downloaded a tarball with that exact name.
If not, then wget it (and proceed to validate its md5sum). When we wget it,
use '--timestamping' to affirm that we definitely want the server's timestamp,
just because that's inherently desirable--for instance, I can look in my
cached-downloads directory and see that my FOP tarball is dated 2003, and that
alone is a good enough reason, though there may be others.

> It would seem to be useless then as time stamping is to determine whether
> the file needs to be updated from server and here it's always the case. I

Okay, the meaning you cite for '--timestamping' seems to be the usual one,
but I actually don't care about that. For me, what matters is getting the
server's timestamp, which you might consider a side effect. I don't want the
"primary" effect of exiting early if everything matches (in which case
(because of '[ -e $@ ]' preceding) the 'wget' command isn't even reached).

Why don't I just do
-       cd $(cache_dir) && [ -e $@ ] || $(WGET) $(WGETFLAGS) $(address@hidden)
+       cd $(cache_dir) &&              $(WGET) $(WGETFLAGS) $(address@hidden)
? Because that seems more fragile. Most significantly, when sf.net is one of
the servers we deal with, the simpler command might fail because sf.net is
down (again). A less likely but possible problem is that a previous failure
caused a truncated or even zero-byte '$@' to exist in the cached directory,
with a later timestamp than the server. That's an argument in favor of
retaining the md5sum test, for now at least, even though yesterday we
talked about dropping it for '.zip' archives from GitHub.

> thought it was just to give the correct timestamp to the file, but this is
> already the case by default, you have to use --no-use-server-timestamps
> explicitly to change this.

The manpage, e.g.,
  http://manpages.ubuntu.com/manpages/gutsy/man1/wget.1.html
doesn't seem to specify that it's a default. And the gnu manual is
equivocal, e.g.:
  http://www.gnu.org/software/wget/manual/html_node/Time_002dStamping-Usage.html
| time-stamping info is preserved locally, even without ‘-N’ (at least for 
HTTP).

>  The reason I looked into this was the incompatibility of --timestamping
> with --output-document (which makes sense). I spent some time thinking
> about how I could still use --timestamping even with the latter before
> realizing that it shouldn't be used at all.

I haven't yet been able to understand why we need '--output-document' for the
'.zip' archive from GitHub. Let's see...I'll add this to the 'foo' target in
'install_wx.make' with the patch I posted last night (note the ".X" added to
the name of the archive retrieved with '--output-document'):

        cd $(cache_dir) && $(WGET) $(WGETFLAGS) 
--output-document=$(wx_archive).X $($(wx_archive)-url)
        cd $(cache_dir) && $(WGET) $(WGETFLAGS) $($(wx_archive)-url)

and run it:

/lmi/src/lmi[0]$make -f install_wx.make foo
[...]
WARNING: timestamping does nothing in combination with -O. See the manual
for details.

Results:

/lmi/src/lmi[0]$ls -o /cache_for_lmi/downloads/wxWidgets-3.0.0.tar.bz2*
-rw-r--r-- 1 earl 19815734 Nov 12  2013 
/cache_for_lmi/downloads/wxWidgets-3.0.0.tar.bz2
-rw-r--r-- 1 earl 19815734 Aug  5 14:54 
/cache_for_lmi/downloads/wxWidgets-3.0.0.tar.bz2.X

/lmi/src/lmi[0]$md5sum /cache_for_lmi/downloads/wxWidgets-3.0.0.tar.bz2*
241998efc12205172ed24c18788ea2cd 
*/cache_for_lmi/downloads/wxWidgets-3.0.0.tar.bz2
241998efc12205172ed24c18788ea2cd 
*/cache_for_lmi/downloads/wxWidgets-3.0.0.tar.bz2.X

I don't get it. Why do we want '--output-document'? Isn't it better without 
that?

> And now I think it should be
> removed from the places where it is currently used just to avoid such head
> scratching in the future.

I prefer to retain it for the reasons above, and to add a line of explanation
that references this email.