Re: More control over 'make dist'

[Michal Privoznik]

On 14.09.2016 21:35, Eric Blake wrote:
> [re-adding Michal; not sure he is a subscriber. It's usually best to
> reply-to-all rather than assume that all readers are on-list]

Thank you, I'll use your e-mail to reply to previous ones too.

> 
> On 09/14/2016 02:10 PM, Warren Young wrote:
>> On Sep 14, 2016, at 7:49 AM, Peter Rosin <address@hidden> wrote:
>>>
>>> On 2016-09-14 11:33, Michal Privoznik wrote:
>>>>
>>>> ln -s $xml.in $xml.out
>>>>
>>>> However, I was looking into archive produced by 'make dist' the other
>>>> day and found out that the symlinks are not preserved.
>>>
>>> I believe that is for the benefit of supporting unpacking the release
>>> tarball on systems that do not support symlinks, or where symlinks are
>>> not as flexible as one might wish for.

Well, I believe it's project developers to say whether they want to
support it or not. For instance if there's a project implementing a
linux kernel module I think it's fair to expect some reasonable
environment too. Moreover there are just a few filesystems that don't
support symlinks (including FAT which would be dead already if it wasn't
for all the portable electronics).

>>
>> The question, then, is whether libvirt would ever be unpacked on such a 
>> system.
>>
>> I’m barely aware of what libvirt does, but I think I can come up with a 
>> plausible scenario: libvirt built from source on Cygwin.
>>
>> It appears from the home page that libvirt already supports Hyper-V, so a 
>> naive user might decide to build it under Cygwin rather than whatever native 
>> Windows toolchain is currently used for that case.  (The reason being that 
>> libvirt, coming from the Linux world, probably builds better under a 
>> Unix-like environment.)  Since NTFS symlinks have a number of unfortunate 
>> limitations[1] and restrictions[2], the tarball almost certainly won’t 
>> unpack correctly.

Well, we've stopped providing official .exe a long ago (2011), but we
still allow users to compile client library using mingw (no tests are
run there though). But still - it is build under Linux with some
reasonable filesystem (i.e. one supporting symlinks).

> 
> Cygwin unpacks and handles symlinks just fine as one of the many things
> it emulates, so that's not the issue (note, however, that non-cygwin
> apps are generally unable to recognize cygwin symlinks, which are
> intentionally NOT done as NTFS symlinks because NTFS symlinks are not
> POSIXy).
> 
> I'd be more worried if trying to unpack libvirt and build it with mingw,
> where you no longer have cygwin symlinks in the mix, and would indeed be
> limited to just NTFS symlinks.  But I think that mingw ports of tar
> already take care of that, by turning symlinks in the tarball into file
> copies (where the link target was also in the tarball) or an error
> message (where the link target is unknown) (although I haven't actually
> tested what MSYS would actually do, so I welcome further comments from
> anyone with experience).
> 
> And yes, libvirt is (cross-)built for mingw on Fedora already, as well
> as me getting ready to prepare a Cygwin distribution build of libvirt
> (both for the remote machine control aspect, and for the HyperV aspect).
>  I'm less familiar with cross-building for mingw to know for certain
> whether symlinks in tarballs cause problems for mingw targets, or
> whether Wine emulation on Linux even handles symlinks.  So that agrees
> with the notion of making tarballs as portable as possible by avoiding
> symlinks.
> 
> On the other hand, maybe the GNU Coding Standards should be updated to
> state that avoiding symlinks is no longer required of GNU packages (it's
> entirely plausible that when automake first coded up the tar -h rule,
> years ago, there really were Unixy systems where either the file system
> or the tar program didn't handle symlinks as gracefully as desired,
> which is a different beast than today's mingw situation).
> 

Agreed. I'm not advocating for every project out there using it, I just
think that it would be nice if developers are given the choice.

Michal