[Qemu-devel] ANNOUNCE: libnbd 1.0 & nbdkit 1.14 - high performance NBD client and server

[Richard W.M. Jones]

I'm pleased to announce the joint release of libnbd 1.0 and nbdkit 1.14.
These are a high performance NBD client library and server.


Key features of libnbd:

 * Synchronous API for ease of use.
 * Asynchronous API for writing non-blocking, multithreaded clients.
   You can mix both APIs freely.
 * High performance.
 * Minimal dependencies for the basic library.
 * Well-documented, stable API.
 * Bindings in several programming languages.
 * Shell (nbdsh) for command line and scripting.

Git: https://github.com/libguestfs/libnbd
Download: http://download.libguestfs.org/libnbd/1.0-stable/
Fedora: https://koji.fedoraproject.org/koji/packageinfo?packageID=28807


Key features of nbdkit:

 * Multithreaded NBD server written in C with good performance.
 * Minimal dependencies for the basic server.
 * Liberal license (BSD) allows nbdkit to be linked to proprietary
   libraries or included in proprietary code.
 * Well-documented, simple plugin API with a stable ABI guarantee.
   Lets you export “unconventional” block devices easily.
 * You can write plugins in C, Lua, Perl, Python, OCaml, Ruby, Rust,
   shell script or Tcl.
 * Filters can be stacked in front of plugins to transform the output.

Git: https://github.com/libguestfs/nbdkit
Download: http://download.libguestfs.org/nbdkit/1.14-stable/
Fedora: https://koji.fedoraproject.org/koji/packageinfo?packageID=16469

Release notes for nbdkit 1.14:

Server performance improvements: Nagle's algorithm is disabled;
instead MSG_MORE is used to control when outgoing packets are sent.
Ramdisk plugins now support more efficient zeroing.  (Eric Blake).

Plugins can now select their thread model at run time.  Thread model
selection is not finalized until after the config stage (Eric Blake).

The server supports NBD_CMD_CACHE for prefetching.  Plugins and
filters have been updated to implement this where it makes sense (Eric
Blake).

Low level pthread lock failures now call abort().  This should never
happen in real situations (Eric Blake).

The server will not advertize multi-conn support if the internal
thread model is serialized, avoiding a possible deadlock in some
clients (Eric Blake).

New server option ‘--no-sr’ can be used to disable structured replies
(Eric Blake).

The server will now refuse to start if stdin/stdout/stderr are closed,
to avoid potential issues with file descriptor handling in plugins
(Eric Blake).

‘$uri’ can be used to specify an NBD URI in ‘--run’ parameters (Eric
Blake).

New ‘stats’ filter displays elapsed statistics about NBD operations,
such as the number of bytes read and written.

New ‘nocache’ filter which disables cache requests, used to determine
how effective they are (Eric Blake).

New ‘noparallel’ filter which can be used to override the plugin’s own
choice of thread model.  Used to determine how the thread model
affects performance, or to serialize plugins if required (Eric Blake).

New ‘cacheextents’ filter to cache extents requests, especially useful
with VDDK which has a slow implementation of extents (Martin
Kletzander).

In the ‘full’, ‘memory’, ‘null’, ‘pattern’ and ‘random’ plugins, the
size= prefix can be omitted, ie: nbdkit memory 1G (instead of size=1G)

The ‘nbd’ plugin has multiple enhancements: It may be built using
libnbd instead of constructing raw NBD packets; NBD_OPT_GO is
supported; add structured reads; implement NBD_CMD_BLOCK_STATUS;
support TCP sockets; forward NBD_CMD_CACHE; retry connections; shared
connections; magic ‘uri’ parameter; TLS support (Eric Blake).

The ‘vddk’ plugin now reports hole extents correctly when using the
‘single-link’ flag (Martin Kletzander).

The ‘cache’ and ‘cow’ filters now avoid copying data through a bounce
buffer when it is already sufficiently aligned (Eric Blake).

Filters (such as the delay and rate filter) which sleep no longer
cause long delays when the server is shut down (Eric Blake).

Multiple fixes to the ‘rust’ plugin (Martin Kletzander).

Multiple enhancements and clean ups to the test suite which should
make tests better and more reliable.

OCaml plugins can now use ‘parse_size’, ‘parse_bool’, ‘read_password’
calls, and there is also a new example plugin.

On platforms which lack atomic O_CLOEXEC support the most serialized
thread model is always selected to avoid leaking file descriptors
(Eric Blake).



-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-builder quickly builds VMs from scratch
http://libguestfs.org/virt-builder.1.html