|
| From: | Tim Rühsen |
| Subject: | Re: [Wget-dev] wget2 | WIP: Add c-ares support for quickly fill DNS cache in background (!444) |
| Date: | Mon, 26 Aug 2019 15:03:51 +0000 |
Tim Rühsen commented on a discussion: https://gitlab.com/gnuwget/wget2/merge_requests/444#note_208172748 The point is that someone came over with 300 millions of domains. He wanted to scan the homepages (or whatever). I gave it a try and wondered why my network bandwidth wasn't fully used. If you increase --max-threads over a certain limit, you'll run into process/kernel limits. I didn't want to go into kernel tuning. So with the right settings we are at ~1.500 DNS lookups per second - if we can reach 5000/s we wouldn't have DNS being a bottleneck. -- Reply to this email directly or view it on GitLab: https://gitlab.com/gnuwget/wget2/merge_requests/444#note_208172748 You're receiving this email because of your account on gitlab.com.
| [Prev in Thread] | Current Thread | [Next in Thread] |