Re: Let BIO also handle already allocated buffers

[Alessio Vanni]

Christian Grothoff <address@hidden> writes:

> However, there are two issues. First, your patch introduces a new type
> 'struct GNUNET_BIO_MemoryHandle'. I think to avoid having to duplicate
> all of the individual BIO-read/write functions

Yes, I made a new struct to avoid touching already-existing code.
Mostly, it was because I didn't want to "break" the API.  Anyway, it's
not really an important thing, so if the concept of having BIO work on
memory buffers is ok, a future patch can simply use the existing types.

> more similar to the gnunet{pq,sq,my,json} APIs we have that use similar
> 'spec' arrays to serialize from/to database and or JSON.

About this, I think it's very verbose compared to the current syntax.
Consider the following (without error checking):

struct GNUNET_BIO_ReadSpec rs[] = {
    GNUNET_BIO_read_spec_int32 (&num),
    GNUNET_BIO_read_spec_end (),
};

GNUNET_BIO_read (handle, rs);

if (0 < num) {
    struct GNUNET_BIO_ReadSpec rs2[] = {
        GNUNET_BIO_read_spec_string (&string),
        GNUNET_BIO_read_spec_end (),
    }

    GNUNET_BIO_read (handle, rs2);
} else {
    struct GNUNET_BIO_ReadSpec rs2[] = {
        GNUNET_BIO_read_spec_int32 (&num2),
        GNUNET_BIO_read_spec_int32 (&num3),
        GNUNET_BIO_read_spec_int32 (&num4),
        GNUNET_BIO_read_spec_end (),
    }

    GNUNET_BIO_read (handle, rs2);
}

This can happen e.g. when dealing with messages between client and
services.  Of course, with enough cleverness it can probably be
optimized to make it less verbose, but I think the I/O itself is too
"dynamic" for this kind of "declarative" syntax, unlike database access
which is often based on well-known queries.

In my opinion, a nicer approach could be something based on Java's
Buffer object from the NIO package.  For example (again no error
checking; also the names could use some more work):

struct GNUNET_BIO_Buffer *b1 = GNUNET_BIO_create_from_file ("myfile");
struct GNUNET_BIO_Buffer *b2 = GNUNET_BIO_create_from_buffer (some_data, 
data_length);

GNUNET_BIO_write_string (b1, "mystring");
GNUNET_BIO_write_int32 (b2, 32);

GNUNET_BIO_flip (b1);
GNUNET_BIO_flip (b2);

GNUNET_BIO_read_string (b1, &str);
GNUNET_BIO_read_int32 (b2, &num);

Another solution could simply be to keep BIO to handle I/O on files and
move the various de/serialization functions to a separate library, so we
would get something like:

struct GNUNET_BIO_WriteHandle *wh = GNUNET_BIO_write_open ("myfile");
struct GNUNET_DISK_FileHandle *fh = GNUNET_DISK_file_open ("myfile", 
GNUNET_DISK_OPEN_WRITE, perms);
struct GNUNET_FIO_Serialized *s = GNUNET_FIO_serialize_string (str);

GNUNET_BIO_write (wh, s);
GNUNET_DISK_file_write (fh, s->data, s->size);
GNUNET_memcpy (mybuffer, s->data, s->size);

What do you think?

Thanks,
A.V.