wget2 | wget2 exits with 8 when the robots.txt job redirects (302) and the subsequent job receives HTTP error 404 (#677)

[@omos]

Ondrej Mosnáček created an issue: https://gitlab.com/gnuwget/wget2/-/issues/677



It seems the code doesn't handle this situation well and treats the second 
(redirected) job differently, so it ends up doing 
`set_exit_status(EXIT_STATUS_REMOTE)` in this case, even though without the 
redirect a 404 error would be ignored for robots.txt (if I'm reading the code 
right).

To reproduce this, you can try to recursively download files from e.g. 
`https://download.fedoraproject.org/pub/fedora/linux/releases/40/Cloud/x86_64/images`
 (though the behavior may differ depending on the mirror that 
`download.fedoraproject.org` picks for you):
```
wget2 -nd -nH -H -r -l 1 -A '*-CHECKSUM' 
https://download.fedoraproject.org/pub/fedora/linux/releases/40/Cloud/x86_64/images
```
The above command fails with the return code 8, even though the only failure 
seems to be from the robots.txt fetching.

-- 
Reply to this email directly or view it on GitLab: 
https://gitlab.com/gnuwget/wget2/-/issues/677
You're receiving this email because of your account on gitlab.com.