chicken-hackers
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[PATCH][simple-md5] Enable hashing of big files


From: lou
Subject: [PATCH][simple-md5] Enable hashing of big files
Date: Mon, 16 Dec 2024 01:19:06 +0100


Hi,

Here's the promised follow-up to my earlier memory-mapped-files
patch.

Enabling simple-md5 to digest big files by splitting files that
are bigger than what the C implementation can digest into chunks
and recuring over them.

Two notes:

 1. This implementation is really only useful for posix
    environments. The alternative `mapped-pointer` implementation
    for Windows means, as far as I understand, that the Windows
    version is limited by physical memory. I don't have a Windows
    machine, someone who does should do some testing on whether
    the divergence is needed. Nominally `memory-mapped-files`
    implements a mmap equivalent for Windows, so it should be
    possible to just use the posix code unconditionally.

    Putting the `mapped-pointer` call inside the recursion,
    mapping the individual chunks, would work around this as
    well, but result in an unnecessary amount of mmaps on posix
    systems.

 2. I hope I got the types right. I got a bit confused because
    `unsigned-integer` seems to only accept values up to INT_MAX
    not up to UINT_MAX, which wasn't clear to me from the
    documentation here:
    http://wiki.call-cc.org/man/5/Foreign%20type%20specifiers#integers

I'll take a look at simple-sha1 in a couple of days, I suspect it
would benefit from a similar patch but I haven't checked that out
yet. Hope this is useful to some people.

~~Lou

Attachment: 00-chicken-simple-md5-hash-big-files.patch
Description: Text Data


reply via email to

[Prev in Thread] Current Thread [Next in Thread]