[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Wget-dev] wget2 | --robots=off / --no-robots should still download the

From: Tim Rühsen
Subject: [Wget-dev] wget2 | --robots=off / --no-robots should still download the robots.txt file and scan it for sitemaps (#456)
Date: Wed, 24 Jul 2019 08:24:11 +0000

Tim Rühsen created an issue:

  See also https://savannah.gnu.org/bugs/?56660

Reply to this email directly or view it on GitLab: 
You're receiving this email because of your account on gitlab.com.

reply via email to

[Prev in Thread] Current Thread [Next in Thread]