Re: wget2 | Fix robots.txt parser (!510)

[Avinash Sonawane (@rootkea)]

Avinash Sonawane commented:


Also, as per `fuzz/README.md` we need to `./get_all_corpora` periodically. May 
be we can automate it using CI?

I'm thinking a separate CI job which downloads the corpora and then raises the 
MR against this very repo. We can use a dummy Gitlab account to [create 
MR](https://docs.gitlab.com/ee/user/project/push_options.html#push-options-for-merge-requests)
 from CI. Clearly we need to guard such a job (maybe using branch name or 
username?) to avoid the MR recursion!

Or maybe running `./get_all_corpora` periodically is not much of an issue? 
Anyways, since `google-cloud-cli` (which provides `gsutil` needed to get the 
corpora) is not is Debiab repos I won't be able to help with the corpora 
fetching.

-- 
Reply to this email directly or view it on GitLab: 
https://gitlab.com/gnuwget/wget2/-/merge_requests/510#note_1044752288
You're receiving this email because of your account on gitlab.com.