Re: feature request for coreutils: b2sum

[Pádraig Brady]

On 27/05/15 18:59, Zooko Wilcox-OHearn wrote:
> Dear coreutils folks and RMS:
> 
> I'm writing to ask you to add the BLAKE2 hash function to coreutils.
> 
> The reason is that md5sum is the most widely-used tool for file
> integrity, but it has flaws that could lead to users being exploited
> if they use md5sum in certain reasonable-sounding ways. (I refer to
> the length-extension and collision attacks on MD5.)
> 
> In addition to those known vulnerabilities, MD5 is widely considered
> to be "weak" by cryptographers, which is reason to worry that new
> vulnerabilities could be discovered in it.
> 
> Another alternative is sha256sum, but it takes a lot more CPU to
> compute, and most users never upgrade to a more secure but
> worse-performing tool, apparently. Also sha256sum remains vulnerable
> the length-extension problem, although not to the collision attacks.
> 
> A newer alternative is SHA3. It fixes both the length-extension and
> collision vulnerabilities, but like sha256sum, it requires a lot more
> CPU than md5sum does.
> 
> I'm requesting that you add b2sum, which is the BLAKE2 algorithm. I'm
> one of the authors of that algorithm. The appeal of it is that it is
> as secure as SHA-3, as far as we know, but it is *faster* than MD5.
> 
> Here's my blog post extolling the virtues of BLAKE2 as a secure hash 
> algorithm:
> 
> https://leastauthority.com/blog/BLAKE2-harder-better-faster-stronger-than-MD5.html
> 
> Below is the timings of me hashing a 1 GB file on my quad-core Intel
> Core i5-3427U CPU @ 1.80GHz, using md5sum, sha256sum, b2sum, and then
> b2sum in an optional parallel mode which uses multiple cores.
> 
> There is now an RFC for blake2:
> https://datatracker.ietf.org/doc/draft-saarinen-blake2/?include_text=1
> 
> In sum, by adding b2sum to coreutils, we can help users protect
> themselves from attackers, and also have a more efficient tool for
> file integrity. You can get the source code here:
> https://blake2.net/#dl
> 
> Regards,
> 
> Zooko
> 
> Timings:
> 
>  $ time md5sum randfile.0
> 4ac1b3ded5072c83ab877f284cc61420  randfile.0
> 
> real    0m2.126s
> user    0m1.960s
> sys     0m0.165s
>  $ time sha256sum randfile.0
> c1bd0be4c1c1e480f288f6ad2b70620b914ebb9047e693af6c085d4f60f2979c  randfile.0
> 
> real    0m5.721s
> user    0m5.562s
> sys     0m0.156s
>  $ time b2sum randfile.0
> 342045493847534951732e8be4986b6737c98e9892954ddeee90463796afd9b21d57d606c2c39ebc65da04ea82523971f2846f4d471d05e4790988fb17d4788b
> randfile.0
> 
> real    0m1.808s
> user    0m1.682s
> sys     0m0.126s
>  $ time b2sum -a blake2sp randfile.0
> 67cc72d895a2efa5c129ba3597707069999e1efc6f4d7822263703ee6039435d randfile.0
> 
> real    0m1.124s
> user    0m3.413s
> sys     0m0.217s

I see blake was an SHA-3 finalist, and this is blake2.
BTW we were thinking about adding sha3sum also:
http://lists.gnu.org/archive/html/coreutils/2012-12/msg00090.html
Given the increasing number of hash functions
I'm leaning a little towards a single utility with a --hash=blake2 etc.
BTW on MD5 timings we can use the optimized openssl routines
where available, so the best comparison is against `openssl md5 $file`

thanks,
Pádraig.