Re: thoughts on ports

[Mark H Weaver]

Hi Andy,

Andy Wingo <address@hidden> writes:
> I have been thinking about ports recently.  I know other folks have had
> some thoughts here too, so it's probably good to have a discussion about
> how they should look.

I apologize that I've not yet had time to properly look into your eports
work, so I'm afraid this email will not address your points adequately,
but nonetheless I've been thinking about the ports code sporadically for
many months, and here are my preliminary (and perhaps controversial :)
thoughts:

First of all, I think the ports code should be split into two layers:
the lower layer should implement only byte ports (i.e. binary I/O).
This layer should have multiple backends to support POSIX-style ports
(e.g. file and socket I/O) as well as bytevector ports and user-defined
"soft" byte ports.

The upper layer should instead offer a character-based interface
(probably unicode code-points in practice).  It should also support
multiple backends, including string ports, user-defined "soft" character
ports, and perhaps most importantly a transcoding port that provides a
character-based view of any arbitrary byte port, using a particular
encoding.

IMO, port-encoding should not be a fundamental property of all ports.
It should _only_ be a property of transcoding ports.  For example,
SRFI-6 string ports should not have any encoding, nor should
user-defined "soft" character ports.  Transcoding ports should also
support mixed byte/character-based I/O.

If users want to do mixed-I/O on a string port, they can convert it to a
bytevector in the desired character encoding and then create a
transcoding bytevector port.

One more recommendation for efficient implementation: internal routines
should always work with _blocks_ of bytes or characters, rather than
individual bytes or characters.  Our current "soft" ports are hopelessly
efficient for this reason, and they cannot be fixed without changing
their interface, so they should be deprecated and replaced.

Regarding the idea of moving some of this code to Scheme, it sounds like
a great idea when we have good native code generation on the important
upcoming platforms (x86/arm/mips at the very least), but until then I
think our ports code better stay in C, but with hooks to implement ports
in Scheme, analogous to our support for extensible numerics.

Nonetheless, I like the idea of using bytevectors for buffering in byte
ports, and offsets instead of pointers.

What do you think?

     Best,
      Mark