|
From: | Tim Rühsen |
Subject: | [Wget-dev] wget2 | --robots=off / --no-robots should still download the robots.txt file and scan it for sitemaps (#456) |
Date: | Wed, 24 Jul 2019 08:24:11 +0000 |
Tim Rühsen created an issue: See also https://savannah.gnu.org/bugs/?56660 -- Reply to this email directly or view it on GitLab: https://gitlab.com/gnuwget/wget2/issues/456 You're receiving this email because of your account on gitlab.com.
[Prev in Thread] | Current Thread | [Next in Thread] |