|
| From: | Archit Pandey |
| Subject: | Re: [Wget-dev] wget2 | --robots=off / --no-robots should still download the robots.txt file and scan it for sitemaps (#456) |
| Date: | Mon, 21 Oct 2019 16:42:59 +0000 |
Archit Pandey commented: Hello @rockdaboot, I'll be interested to take this up. I'm currently looking into the code that needs to be changed. -- Reply to this email directly or view it on GitLab: https://gitlab.com/gnuwget/wget2/issues/456#note_233445305 You're receiving this email because of your account on gitlab.com.
| [Prev in Thread] | Current Thread | [Next in Thread] |