Re: Guile Binary Format 0.1

[Keisuke Nishida]

At Thu, 22 Feb 2001 04:23:54 +0100 (MET),
Dirk Herrmann wrote:
> 
> Hello folks,
> 
> with the current implementation of writing objects in a binary manner, we
> can not solve the following problem:
> 
> (define foo (cons 'a 'b))
> (define bar (cons 'c foo))
> (define baz (cons 'd foo))
> (eq? (cdr bar) (cdr baz))
>   ==> #t
> (binary-write bar some-port)
> (binary-write baz some-port)
> ... and later:
> (define newbar (binary-read previously-written-data))
> (define newbaz (binary-read previously-written-data))
> (eq? (cdr newbar) (cdr newbaz))
>   ==> #f
> 
> The problem is, that some state would have to be preserved between
> consecutive write and read operations.  In the extreme, this could mean
> that every single object that is written or read from a binary file has to
> be kept in a lookup table until the file is completely written or read.
> When writing, things are not that bad, because weak lookup tables can be
> used:  An object that is not referenced any more can not be part of 
> future written objects, which means that for writing a weak table is OK.
> But, when reading a binary file no one can tell whether some object that
> was read at some time before will be referenced in the file later on.

My plan is to associate a lookup table with the port during reading/
writing.  The same table is used through a consecutive read or write,
so there is no problem.  When the port is collected, the table is
corrected as well.

There are other considerations about binary operations.  When I think
about storing several modules in a single file, I realize that the
current format lacks an ability of selective loading; that is, we may
want to load a specific module at a time instead of loading all
modules at once.  I need to think about it.

> Besides these difficulties, a way to achieve this in an implementation
> could be to make a move from the current operations 'binary-read' and
> 'binary-write' to something that is a binary-port, and then just use
> 'read' and 'write', which are automatically performed as binary reads and
> writes, since the port's implementation will take care of that.
> 
> A problem with this approach is, that for a binary port an operation like
> read-char does not make sense.  Thus, if we want to go that way, we would
> have to generalize the current idea of what a port is:  If I understand
> things correctly, ports currently have to be implemented using character
> based buffers.  For some objects that can be seen as ports this is too
> restrictive.  Beside the binary ports, directory streams are a good
> example, which are also not realised as port objects currently: Reading
> single characters from directory streams does not make sense either.
> 
> Thus, a long term goal could be to define a port interface that offers
> more freedom when it comes to implementing other types of ports.  This
> would also help to unify a couple of objects types in guile, like
> directory streams, ports with print state and the ordinary ports.
> However, definition of a generic but still efficient port type system
> seems to be quite difficult.  Thus, no suggested solution by me, only the
> suggestion to keep this topic in mind...

Conceptually, I think, ports are interface of reading/writing objects.
The representation of a port buffer is always a binary stream, but how
to store objects in the buffer can be different.  We could specify the
format as follows:

  (define port (current-output-port))

  (set-port-format! port 'human-readable)
  (write #\a port)                 -> #\a
  (write 'foo port)                -> foo

  (set-port-format! port 'character-stream)
  (write #\a port)                 -> a
  (write 'foo port)                ;; ERROR: Not a character

  (set-port-format! port 'binary-object)
  (write #\a port)                 -> address@hidden
  (write 'foo port)                -> address@hidden@#$!foo\0

  (define port (opendir "/"))
  (set-port-format! port 'directory-stream)
  (read port)                      -> "."

Now, we can define the following functions:

  (define (write-char ch port)
    (let ((orig (port-format port)))
      (set-port-format! port 'character-stream)
      (write ch port)
      (set-port-format! port orig)))

  (define (binary-write obj port)
    (let ((orig (port-format port)))
      (set-port-format! port 'binary-object)
      (write obj port)
      (set-port-format! port orig)))

So, in this sense, we can justify the existence of binary-write...

Kei