[bug #66338] fails to fetch files that redirect from a directory (workaround: use aria2c)

[Eo Koochu]

URL:
  <https://savannah.gnu.org/bugs/?66338>

                 Summary: fails to fetch files that redirect from a directory
(workaround: use aria2c)
                   Group: GNU Wget
               Submitter: eokoochu
               Submitted: Wed 16 Oct 2024 10:10:53 AM UTC
                Category: Program Logic
                Severity: 3 - Normal
                Priority: 5 - Normal
                  Status: None
                 Privacy: Public
             Assigned to: None
         Originator Name: 
        Originator Email: 
             Open/Closed: Open
         Discussion Lock: Any
                 Release: None
        Operating System: GNU/Linux
         Reproducibility: Every Time
           Fixed Release: None
         Planned Release: None
              Regression: None
           Work Required: None
          Patch Included: No


    _______________________________________________________

Follow-up Comments:


-------------------------------------------------------
Date: Wed 16 Oct 2024 10:10:53 AM UTC By: Eo Koochu <eokoochu>
When an URL refers to a directory that redirects to a file, wget gives up
instantly without checking the redirect header. This is a sample broken
session:

===8<------------------------------
$ wget
'https://consumer.georgia.gov/document/document/consumer-complaint-form-english/download'
--[timestamp]-- 
https://consumer.georgia.gov/document/document/consumer-complaint-form-english/download
Resolving consumer.georgia.gov (consumer.georgia.gov)... 104.18.74.48
Connecting to consumer.georgia.gov (consumer.georgia.gov)|104.18.74.48|:443...
connected.
HTTP request sent, awaiting response... 200 OK
Length: 183944 (180K) [application/pdf]
download: Is a directory
===8<------------------------------

This happened in version 1.21.3. It may be a similar bug to this one (not
sure):

https://savannah.gnu.org/bugs/?func=detailitem&item_id=45992

I did not experiment with the --trust-server-names option. But if that’s a
factor then wget still needs to more accurately and thoroughly inform the
user, in which case it’s arguably a defect nonetheless. E.g. it should say
in the error msg something like “not following redirect for security
reasons”, if that’s the case.

**workaround**

The workaround is basically to use anything other than wget. Firefox works but
FF suggests the wrong filename. Aria2c works flawlessly and sets the correct
filename. This is what the aria2c session looks like:

===8<------------------------------
$ aria2c
'https://consumer.georgia.gov/document/document/consumer-complaint-form-english/download'
[timestamp] [NOTICE] Downloading 1 item(s)
[#6234b1 0B/0B CN:1 DL:0B]
[timestamp] [NOTICE] Download complete: $PWD/Consumer Complaint Form -
2024-09-30.pdf

Download Results:
gid   |stat|avg speed  |path/URI
======+====+===========+=======================================================
6234b1|OK  |   459KiB/s|$PWD/Consumer Complaint Form - 2024-09-30.pdf

Status Legend:
(OK):download completed.
===8<------------------------------









    _______________________________________________________

Reply to this item at:

  <https://savannah.gnu.org/bugs/?66338>

_______________________________________________
Message sent via Savannah
https://savannah.gnu.org/

signature.asc
Description: PGP signature