bug-wget
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Bug report] Wget doesn't respect --https-only option


From: Vipul
Subject: [Bug report] Wget doesn't respect --https-only option
Date: Wed, 6 May 2020 01:52:23 +0000

Hi,

Recently, I noticed a weird behavior of Wget, whose detailed description
as follows,

### Description
When I use `--https-only` option with Wget to force download over https;
it still use http protocol with non-https link and https link following
non-https link, instead of throwing an error message, like Wget2.


### Steps to reproduce
- Run following command
    - A non-https link
        $ wget --no-config -d --server-response --https-only 
"http://example.com";       

    - A https link following non-https
        $ wget --no-config -d --server-response --https-only
"https://www.gutenberg.org/ebooks/29765.txt.utf-8";


### Actual output
Wget perform download over non-https protocol.


### Expected output
Wget should throw an error message.


### Additional
I cannot reproduce this behavior on Wget2.


### System info
- Wget
GNU Wget 1.20.1 built on linux-gnu.

- OS
Debian 10 (Buster).


### Debug log
#### A non-https link
$ wget --no-config -d --server-response --https-only "http://example.com";

Setting --server-response (serverresponse) to 1
Setting --server-response (serverresponse) to 1
Setting --https-only (httpsonly) to 1
Setting --https-only (httpsonly) to 1
DEBUG output created by Wget 1.20.1 on linux-gnu.

Reading HSTS entries from /home/finn/.wget-hsts
URI encoding = ‘UTF-8’
Converted file name 'index.html' (UTF-8) -> 'index.html' (UTF-8)
--2020-05-06 06:36:46--  http://example.com/
Resolving example.com (example.com)... 93.184.216.34,
2606:2800:220:1:248:1893:25c8:1946
Caching example.com => 93.184.216.34 2606:2800:220:1:248:1893:25c8:1946
Connecting to example.com (example.com)|93.184.216.34|:80... connected.
Created socket 3.
Releasing 0x0000555fe71ae120 (new refcount 1).

---request begin---
GET / HTTP/1.1
User-Agent: Wget/1.20.1 (linux-gnu)
Accept: */*
Accept-Encoding: identity
Host: example.com
Connection: Keep-Alive

---request end---
HTTP request sent, awaiting response...
---response begin---
HTTP/1.1 200 OK
Age: 473419
Cache-Control: max-age=604800
Content-Type: text/html; charset=UTF-8
Date: Wed, 06 May 2020 01:06:51 GMT
Etag: "3147526947+ident"
Expires: Wed, 13 May 2020 01:06:51 GMT
Last-Modified: Thu, 17 Oct 2019 07:18:26 GMT
Server: ECS (nyb/1D0F)
Vary: Accept-Encoding
X-Cache: HIT
Content-Length: 1256

---response end---

  HTTP/1.1 200 OK
  Age: 473419
  Cache-Control: max-age=604800
  Content-Type: text/html; charset=UTF-8
  Date: Wed, 06 May 2020 01:06:51 GMT
  Etag: "3147526947+ident"
  Expires: Wed, 13 May 2020 01:06:51 GMT
  Last-Modified: Thu, 17 Oct 2019 07:18:26 GMT
  Server: ECS (nyb/1D0F)
  Vary: Accept-Encoding
  X-Cache: HIT
  Content-Length: 1256
Registered socket 3 for persistent reuse.
URI content encoding = ‘UTF-8’
Length: 1256 (1.2K) [text/html]
Saving to: ‘index.html’

index.html
100%[============================================================>]
1.23K  --.-KB/s    in 0s

2020-05-06 06:36:52 (128 MB/s) - ‘index.html’ saved [1256/1256]



#### A https link follows non-https link
$ wget --no-config -d --server-response --https-only
"https://www.gutenberg.org/ebooks/29765.txt.utf-8";

Setting --server-response (serverresponse) to 1
Setting --server-response (serverresponse) to 1
Setting --https-only (httpsonly) to 1
Setting --https-only (httpsonly) to 1
DEBUG output created by Wget 1.20.1 on linux-gnu.

Reading HSTS entries from /home/finn/.wget-hsts
URI encoding = ‘UTF-8’
Converted file name '29765.txt.utf-8' (UTF-8) -> '29765.txt.utf-8' (UTF-8)
--2020-05-06 06:39:04--  https://www.gutenberg.org/ebooks/29765.txt.utf-8
Certificates loaded: 128
Resolving www.gutenberg.org (www.gutenberg.org)... 152.19.134.47,
2610:28:3090:3000:0:bad:cafe:47
Caching www.gutenberg.org => 152.19.134.47 2610:28:3090:3000:0:bad:cafe:47
Connecting to www.gutenberg.org
(www.gutenberg.org)|152.19.134.47|:443... connected.
Created socket 3.
Releasing 0x0000555c4eed1340 (new refcount 1).

---request begin---
GET /ebooks/29765.txt.utf-8 HTTP/1.1
User-Agent: Wget/1.20.1 (linux-gnu)
Accept: */*
Accept-Encoding: identity
Host: www.gutenberg.org
Connection: Keep-Alive

---request end---
HTTP request sent, awaiting response...
---response begin---
HTTP/1.1 302 Found
Server: Apache
Location: http://www.gutenberg.org/cache/epub/29765/pg29765.txt
Content-Type: text/html; charset=iso-8859-1
X-Powered-By: 2
X-Cacheable: NO: beresp.status
Content-Length: 304
Date: Wed, 06 May 2020 01:10:04 GMT
X-Varnish: 1328227108
Age: 0
Via: 1.1 varnish

---response end---

  HTTP/1.1 302 Found
  Server: Apache
  Location: http://www.gutenberg.org/cache/epub/29765/pg29765.txt
  Content-Type: text/html; charset=iso-8859-1
  X-Powered-By: 2
  X-Cacheable: NO: beresp.status
  Content-Length: 304
  Date: Wed, 06 May 2020 01:10:04 GMT
  X-Varnish: 1328227108
  Age: 0
  Via: 1.1 varnish
Registered socket 3 for persistent reuse.
URI content encoding = ‘iso-8859-1’
Location: http://www.gutenberg.org/cache/epub/29765/pg29765.txt [following]
Skipping 304 bytes of body: [<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML
2.0//EN">
<html><head>
<title>302 Found</title>
</head><body>
<h1>Found</h1>
<p>The document has moved <a
href="http://www.gutenberg.org/cache/epub/29765/pg29765.txt"Skipping 97
bytes of body: [>here</a>.</p>
<hr>
<address>Apache Server at www.gutenberg.org Port 80</address>
</body></html>
] done.
URI content encoding = None
Converted file name '29765.txt.utf-8' (UTF-8) -> '29765.txt.utf-8' (UTF-8)
--2020-05-06 06:39:07--
http://www.gutenberg.org/cache/epub/29765/pg29765.txt
Found www.gutenberg.org in host_name_addresses_map (0x555c4eed1340)
Connecting to www.gutenberg.org (www.gutenberg.org)|152.19.134.47|:80...
connected.
Created socket 4.
Releasing 0x0000555c4eed1340 (new refcount 1).

---request begin---
GET /cache/epub/29765/pg29765.txt HTTP/1.1
User-Agent: Wget/1.20.1 (linux-gnu)
Accept: */*
Accept-Encoding: identity
Host: www.gutenberg.org
Connection: Keep-Alive

---request end---
HTTP request sent, awaiting response...
---response begin---
HTTP/1.1 200 OK
Server: Apache
Content-Location: pg29765.txt.utf8
Vary: negotiate,accept-encoding
TCN: choice
Last-Modified: Wed, 13 Mar 2013 13:59:53 GMT
X-Frame-Options: sameorigin
X-Connection: Close
Content-Type: text/plain; charset=utf-8
X-Powered-By: 2
Content-Length: 28956348
Date: Wed, 06 May 2020 01:10:06 GMT
X-Varnish: 1328227142 1328168209
Age: 2264
Via: 1.1 varnish

---response end---

  HTTP/1.1 200 OK
  Server: Apache
  Content-Location: pg29765.txt.utf8
  Vary: negotiate,accept-encoding
  TCN: choice
  Last-Modified: Wed, 13 Mar 2013 13:59:53 GMT
  X-Frame-Options: sameorigin
  X-Connection: Close
  Content-Type: text/plain; charset=utf-8
  X-Powered-By: 2
  Content-Length: 28956348
  Date: Wed, 06 May 2020 01:10:06 GMT
  X-Varnish: 1328227142 1328168209
  Age: 2264
  Via: 1.1 varnish
Disabling further reuse of socket 3.
Registered socket 4 for persistent reuse.
URI content encoding = ‘utf-8’
Length: 28956348 (28M) [text/plain]
Saving to: ‘29765.txt.utf-8’

29765.txt.utf-8                     0%[
                           ]  38.25K  38.5KB/s


Cheers,
Vipul



reply via email to

[Prev in Thread] Current Thread [Next in Thread]