[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
wget2 | Fix robots.txt parser (!510)
From: |
Avinash Sonawane (@rootkea) |
Subject: |
wget2 | Fix robots.txt parser (!510) |
Date: |
Fri, 29 Jul 2022 11:20:10 +0000 |
Avinash Sonawane created a merge request:
https://gitlab.com/gnuwget/wget2/-/merge_requests/510
Project:Branches: rootkea/wget2:robots to gnuwget/wget2:master
Author: Avinash Sonawane
### Approver's checklist:
* [ ] The author has submitted the FSF Copyright Assignment and is listed in
AUTHORS
* [ ] There is a test suite reasonably covering new functionality or
modifications
* [ ] Function naming, parameters, return values, types, etc., are consistent
with existing code
* [ ] This feature/change has adequate documentation added (if appropriate)
* [ ] No obvious mistakes / misspelling in the code
This is a parser conforming to https://www.robotstxt.org/orig.html#format which
also supports
https://datatracker.ietf.org/doc/html/draft-koster-rep#section-2.2 except the
"Allow:" field.
--
Reply to this email directly or view it on GitLab:
https://gitlab.com/gnuwget/wget2/-/merge_requests/510
You're receiving this email because of your account on gitlab.com.
- wget2 | Fix robots.txt parser (!510),
Avinash Sonawane (@rootkea) <=