Re: [GNUnet-developers] [GSoC] Question on "Rust implementation of GNUnet utils" project

[Christian Grothoff]

I think Andrew puts it really nicely here: the choice of an async
solution imposes a particular style on the entire program, which can
result in awkward incompatibilities.


For GNUnet's C code, I strongly believe the choice of going with an
event loop and to not have any threads (but processes) is a good choice.
But even there, discussions with Volker (and Jeff, and others) are
leading towards the fact that the existing scheduler needs to be made
more generic soon.


For Rust, the type system and language features suggest that other
styles might not have the same negative software engineering/security
implications. However, that still doesn't make spawning a fresh thread
for every tiny activity a good thing.

For Java, we made the choice to keep the APIs and programming styles
similar to C (i.e. event loop) to make it easy to switch between one
language without having to relearn how to do everything. However, at the
same time, we of course also tried to keep the code natural in the style
of the Java language.

Given all this, the mio-solution Andrew suggests does seem reasonable,
as it offers keeping the style overall close to the existing GNUnet
design, and avoids spawning threads like crazy. Furthermore, my
understanding is that if we ever do need multiple threads for event
handling, mio supports that nicely.

As for details, like eventual IO vs. rotor, my main problem is that this
entire part of the Rust stack seems to be still experimental or rather
undecided (see https://github.com/rust-lang/rfcs/issues/1081).  The fact
that such a fundamental issue for a new system language is unresolved
(or resolved by redirecting to 3rd party mio which implements it using
"unsafe" constructions) just stresses for me that the best plan is to
have an abstraction: Whatever we decide here we may need to revise for
the next major revision of Rust, so if we had a GNUNET-SCHEDULER-like
*abstraction*, we can likely retarget that logic to match whatever way
Rust evolves to support.  So my suggestion would be to put a
scheduler-like API on top of mio for now, with the language-derived
benefit that it would be thread-safe, even if it doesn't necessarily use
them.


Finally, just to be clear, even with Rust we should preserve the
multi-process architecture for major components. Even outside of
security considerations, the ability to write parts in different
languages is a major benefit that we must not loose.  Also, if we do it
this way, blocking (disk) IO would be less of an issue if it is done in
processes that don't interact directly with the network.


Happy hacking!

Christian
p.s.: My experience with Rust is still very limited, if a Rust crack has
an opinion based on some profound facts or insights into how the Rust
language is likely to evolve, it might be quite possible to sway me to
accept conclusions that differ from what I wrote above.

On 03/21/2016 06:18 PM, Andrew Cann wrote:
> 
> One problem with I have with all these options though is that they seem to
> force a particular paradigm on your entire program. If you're writting an
> application that's fine, but gnunet-rs is a library and I'd like it if the
> users were free to write their gnunet application using state machines, green
> threads, blocking IO or whatever. A vague solution that I'm imagining is if
> there was some way to write a low-level mio-based library in such a way that
> you could then use wrappers that take a generic low-level mio based library 
> and
> promote it to a blocking IO based library, or a coroutine-based library or
> whatever. For now though, I don't have a better description than that.

signature.asc
Description: OpenPGP digital signature