Re: [GNUnet-developers] my TODO list

[Christian Grothoff]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Friday 04 April 2003 11:12 pm, Glenn McGrath wrote:
> Ive thought of a few things id like to do with GNUnet, aimed at
> consolidating/improving existing work rather than adding new features.
>
>
> 1) Remove unnecessary macros.
>    I think its tidier to avoid macros as much as possible, one reason
> being that the compiler cant do type checking on macro, apart from that
> they tend to hide stuff that should be either a function or a variable.

This is just wrong. First, the compiler does still do type checks on the 
marco, the type checking is done after the pre-processor and is still fully 
in effect. So this is no reason against macros. Furthermore, you can not 
replace any of the GNUnet macros with a procedure call that would do the same 
thing since they are all used to add things like __LINE__ and __FILE__ for 
debugging. For example, MALLOC(x) expands to xmalloc(x, __FILE__, __LINE__), 
which of course nobody would want to write by hand. If you do not use a macro 
but do this expansion in a procedure, you will get the wrong line and 
filename. I think we've been very careful to use only a few macros that are 
trivial to understand -- I'm not aware of any one that would make the code 
better if it was replaced by a function. Please give some concrete examples 
to discuss if you still think there is an issue here.

> 2) Make transports a subdir of src/server
>    The transport libraries [udp, smtp, tcp] are only used by gnunetd so
> they could go under the server directory.

They could, but what's the point? A deeply nested tree is not always the 
better choice. I'm happy with where they are, yes, they are conceptually used 
by the server, but the same applies to the application modules which are 
under applications. 

> 3) Configuration options to link to shared rather than dynamic libraries
>    Dynamic loadable libraries are a useful idea in theory, but in
> practice its often not needed.
>    I want to make it a configuration option whether to link specified
> libraries to the binaries as a normal shared library or as a dynamic
> library. I'm not sure how i want to implement it yet, it might get messy
> if there are too many choices.

There are various advantages to the current approach:
* separate compilation and linking: you can recompile the application or
  transport modules independently from the other code, speeding up 
  development (very important for me :-)
* if you build a binary with the library linked in, it will use memory even if
  the user does not want that functionality. Start gnunetd without loading
  the AFS module and compare memory use...
* it's often important to just debug one transport or one application and
  be able to quickly disable all of the others (that may have unexpected
  side-effects), re-running configure with a dozen options to do this is not
  going to make it easier
* what is the problom with dynamic libraries again? Yes, you have to write
  some code to make it happen, but that code is there and I have not had 
  any problems with it so far.

> 4) Cleaner seperation between gnunetd and applications
>
> 4a) Separate config file for gnunetd
>    It seems to me that splitting the configuration file is the easiest
> way to make it simpler, splitting gnunetd config stuff out looks to be a
> good separation point.

I've thought about this, but so far I've decided against it. More files just 
means more things to open and configure for the users, it does not really 
reduce the complexity, in fact it increases it since now you have to know in 
which configuration file the option is that you want to change. 

Also take apache (as an example for a useable, well-designed project), they 
used to have 4 or 5 configuration files that configured different parts of 
the system. These days, they're back to 2 (configuration and mime-types, as 
far as I know). Reducing the number of options to what is useful, writing a 
GUI for configuration or improving the documentation for the options are more 
important in my opinion. 

> 4b) Create a libgnunetd
>    Separate the gnunetd binary from its core functionality, probably
> only have command line argument and config file parsing in the gnunetd
> and the core functionality in the gnunetd library.
>   gnunetd, peer-info, transport check could link to this shared library.

I've thought about this, too. The issue is, what goes into libgnunetd and what 
does not? peer-info and transport-check use very different pieces of the 
gnunetd code. Since for me it's not clear what would be a good boundary 
between libgnunetd and "gnunetd binary only" code, I've not done it so far. 

In terms of cleaning things up, it may make it a lot easier for "third party" 
developers to work on GNUnet code if we would provide installable header 
files such that they can install the gnunet libraries and then add them as a 
dependency, link against them and use the code with some "#include 
<gnunet-header.h>". The current 5-billion-header-files-for-any-small-C file 
approach is not such a great idea for installable header files (it's 
literally impossible to have /usr/include/semaphore.h be gnunet's semaphore 
header, for example). I'm not sure what's the best way to do it, but 
consolidating several header files may be a reasonable thing to do. While I 
generally prefer lots of small, tractable .c and .h files that provide a 
distinct piece of functionality, we should find a way to provide *ONE* header 
file per _exported_ library (exported as in: made available to client code). 
Various approaches that I can think of (cat *.h >> mega_h; one include file 
that includes all the others and placing the others in a subdirectory, moving 
all header-files into one in the CVS) all have their advantages and 
disadvantages, so I'm not sure what's the best way to go. I would suggest 
that if someone wants to give it a try and come up with a suggestion, the 
src/util/ library (libgnunetutil) would be a good first candiate.

> 6) Create a test framework for gnunetd
>    I think the bigest technical hurdle for gnunet is efficient use of
> bandwidth, in particular cutting down unnecessary node-node chatter.
> This chatter is something that will need to be tweaked and tuned as
> GNUnet stabilises.
>    Christian spotted and fixed a problem when he hooked up about 35 (if
> i remember correctly) nodes, it would be easier to find/fix problems if
> we had an easy to setup test framework.

Actually, Eric Hammond is working on a test framework. He's send me a web-page 
with documentation and the code, which I've just put online:

http://www.ovmj.org/GNUnet/archi/

You may want to look into that, Eric is asking in particular for help with 
english, but I guess coding & testing would be useful for him, too :-)

>    It would be good to be able see GNUnet's behaviour with hundreds or
> even thousands of nodes.

For this, I would rather have a simulator than a testbed that is running the 
actual GNUnet code. If you just want to test some large-scale behavior, it 
may get quickly intractable to run the real code. For example, for most 
simulations, there's no point in actually doing the link encryption or 
authentication properly (or at all).

>    A separate libgnunetd would make it easier to create a test
> framework, as each local node could be passed its own config values
> generated dynamically at startup.
>    I do need to think more about this one.

Eric's approach is different (and I think it may be a bit easier).

Christian
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.6 (GNU/Linux)
Comment: For info see http://www.gnupg.org

iD8DBQE+jxdQ9tNtMeXQLkIRAhrsAKChd7tWHIC5a0jgpqZWYkpMWVG6fgCbB+J8
fTGSByXNvI0i3nAlSDbpKGg=
=WZ/j
-----END PGP SIGNATURE-----