wget-dev
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Wget-dev] scripts under /benchmarks


From: Darshit Shah
Subject: Re: [Wget-dev] scripts under /benchmarks
Date: Sat, 22 Sep 2018 09:09:44 +0200
User-agent: NeoMutt/20180716

CCing this mail to the wget-dev list so that the others can see / comment on
it.

I would rather that the scripts are indeed still written in Bash.
The benchmark scripts should be small and generally require shelling out a lot.
Bash is the best fit.

However, right now, I think it a more pressing concern that we have is for
benchmarking the effects of HTTP/2 multiplexing. Look at the discussion here:
https://gitlab.com/gnuwget/wget2/issues/397

The basic idea would be for you to write a script that has Wget2 connect to a
HTTP/2 server. The different scenarios that I can currently think of are:

- Downloading lots of individual files from the server in parallel
- Mirroring the server recursively. Preferably when the recursion requires
  parsing large / complex documents. 

You can start with using a stock nghttp2 server locally. But, you will have to
create the data that it should serve. Also, use `taskset` to set the CPU
affinities of the server and Wget2 client to different CPUs so that script can
measure the CPU utilization as well

* Guy G <address@hidden> [180922 08:59]:
>  Darshit, 
> In parallel to working on the scripts. I would not mind simply re-writing 
> them, be it in Bash or Python. 
> 
> I understand the basic goal -> having an index for comparing between wget, 
> wget2 and curl.
> 1. what are we comparing, speed of downloading content? getting headers?
> 2. we'd like the above to be plotted or other output formats can be 
> acceptable?
> 
> I think if you'll task me with what you need, I may be efficient as well with 
> writing this new (documented) version of a benchmarking component. 
> thank you. 
>    
> 
> 
>   

-- 
Thanking You,
Darshit Shah
PGP Fingerprint: 7845 120B 07CB D8D6 ECE5 FF2B 2A17 43ED A91A 35B6

Attachment: signature.asc
Description: PGP signature


reply via email to

[Prev in Thread] Current Thread [Next in Thread]