Re: [PATCH] Add slice by 8 algorithm to cksum to make it faster.

[Kristoffer Brånemyr]

Hi,
Thanks for testing it on a big endian platform and also fixing the error found 
on that platform! Good to know it calculated the right value at least.

-- 
/Kristoffer Brånemyr 

    Den söndag 27 december 2020 00:55:11 CET, Pádraig Brady <p@draigbrady.com> 
skrev:  
 
 On 26/12/2020 22:54, Pádraig Brady wrote:
> On 26/12/2020 13:53, Kristoffer Brånemyr via GNU coreutils General Discussion 
> wrote:
>> Hi,
>>
>> I modified cksum to use the well known slice by 8 algorithm in the CRC 
>> calculation, to make it faster. On my machine it is several times faster 
>> than the unmodified cksum. It took me a while to figure out since the CRC 
>> calculation in cksum shifts in the opposite direction than most other 
>> implementations I've seen. I would be glad if someone could check this patch 
>> on a big endian machine to see if it produces the correct output! It think 
>> it might, but not sure.
>>
>> You can see the patch here:
>> https://github.com/coreutils/coreutils/pull/43
> 
> Thanks for the patch!
> I wouldn't focus on big endian perf,
> but I will test on a SPARC-Enterprise-T5220 system I have access to.
> 
> I would explicitly depend on byteswap in bootstrap.conf now,
> rather than relying on the transitive dependency through md5sum etc.

A 100MB file improves from 2.50s to 1.80s on the T5220
A 100MB file improves from 0.54s to 0.13s on an i3-2310M

I applied the attached to avoid:

src/cksum.c:201:15:
  error: cast increases required alignment of target type
  [-Werror=cast-align]
  201 |      datap = (uint32_t *)buf;
      |              ^

I'll apply after a bit more testing.

thanks again,
Pádraig