[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
wget2 | Multithreaded download: Change server probing behavior to better
From: |
Long Nguyen (@longnguyen2004) |
Subject: |
wget2 | Multithreaded download: Change server probing behavior to better match browsers (#626) |
Date: |
Thu, 09 Mar 2023 16:22:40 +0000 |
Long Nguyen created an issue: https://gitlab.com/gnuwget/wget2/-/issues/626
Original issue at https://github.com/rockdaboot/wget2/issues/278
Chromium-based browsers, on media content (video embedded with the <video>
tag), send a GET request with `Range: bytes=0-` and discard the body, instead
of a HEAD request. I've seen servers that take advantage of it as a form of
anti-scraping. Specifically, they disallow the use of HEAD requests, and return
404 for requests that don't have a Range header.
I propose that we add `Range: bytes=0-` to every initial GET request, and then
check for 206 or the `Accept-Ranges: bytes` header to determine if segmented
downloading is supported.
--
Reply to this email directly or view it on GitLab:
https://gitlab.com/gnuwget/wget2/-/issues/626
You're receiving this email because of your account on gitlab.com.
- wget2 | Multithreaded download: Change server probing behavior to better match browsers (#626),
Long Nguyen (@longnguyen2004) <=