Re: wget2 | Add tests for large files (size around 2^31 and 2^32bit) (#92)

[@rockdaboot]

Tim Rühsen commented:


Typical corner cases are 32bit vs 64bit variables. If we accidentally use an 
`int` (32bit) where we should have used a `int64_t` we might only 
(automatically) detect this by having a test that creates a 32bit overflow.
Downloading large files seems to be the most common case where we need 64bit 
values, as 32bit can only represent 4GB and we see a growing number of files 
bigger than that.
Such a test would especially be useful on a 32bit system (IMO we don't have one 
in our CI yet).

I would start with a test that simply downloads a single file. Then make sure 
you create this file with exactly 4GB with "random" content (a repetions of say 
a certain number of random bytes seems good). Create a checksum for this file. 
Download the file using wget2. Build the checksum for the downloaded file. If 
both checksums are identical the test succeeded.

Do the same with 4GB +/- 1 byte.

Once this is working we can think about optimizations like building the file 
on-the-fly while downloading (not sure if libmicrohttpd (MHD) allows this. MHD 
is the library we use as HTTP(S) server for testing).

-- 
Reply to this email directly or view it on GitLab: 
https://gitlab.com/gnuwget/wget2/-/issues/92#note_634768590
You're receiving this email because of your account on gitlab.com.