[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [Bug-wget] Timestamping vs incomplete downloads
From: |
Tim Rühsen |
Subject: |
Re: [Bug-wget] Timestamping vs incomplete downloads |
Date: |
Tue, 23 Oct 2018 09:23:03 +0200 |
User-agent: |
Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.2.1 |
On 10/23/18 9:07 AM, Darshit Shah wrote:
> On October 22, 2018 11:49:12 PM UTC, Dave Warren <address@hidden> wrote:
>> Currently when a download with timestamping enabled gets interrupted,
>> the timestamp of the resulting file ends up being the current time and
>> when wget is re-executed after connectivity is restored the local file
>> is then seen as newer and skipped.
>>
>> robocopy handles this a little differently, by setting a date far in
>> the
>> past as a way of ensuring that on a subsequent execution the transfer
>> can be resumed.
>>
>> Is there a better way to handle this situation in wget? A way to force
>> an old date on the file? I'd be happy with a fixed "in the past" date,
>> the service supplied date minus a second, etc. Or some way to detect
>> that the file is incomplete (too small) on a subsequent run?
>
> I haven't tested it but what you say indeed sounds like a valid bug.
>
> The cleanest approach, IMO, is to use the extended file attributes in modern
> systems to store this time at the very beginning and look for it on
> continuation. Setting the time in the past doesn't work since every packet
> that is written will once again update the last modified time. Setting the
> time after each write() is not a feasible solution. What you suggest can only
> work when the client gets a clean exit in the face of an interruption and
> this isn't always the case.
There is an option to skip if-modified-since. With it wget has to make
an extra HEAD request - and that not only returns a file timestamp but
also the length of the file.
@Dave Could you give us an example of a command line where the issue
occurs, resp. could you test with --no-if-modified-since ?
Regards, Tim
signature.asc
Description: OpenPGP digital signature