Re: [GNUnet-developers] speed/load problem while inserting files with gnunet-insert

[Igor Wronsky]

On Wed, 24 Mar 2004, Christian wrote:

> than a few hundred kB. The point is that after processing 300-500kB the
> machine hangs for a moment cause of the disk load. (its quite noisy
> also :)
> It is clear to me that indexing lets say 100Mb takes its time anyway so
> why not do it relaxed in the background without stressing the system.
> Something like reading 100kB and waiting a second for diskcache and
> other applications.

Before starting to blame the kernel on this, what datastore
are you using? I know from experience that atleast
gdbm turns tardy when it fills up.

Perhaps the biggest problem here is that GNUnet datastore
is essentially a random access device for very small blocks.
Even if we'd buffer the blocks in gnunetd for writing,
there still would be no guarantee that something like
gdbm will store them as consequent blocks, and then it'd
just hang or thrash around the moment we flushed the buffer.

I don't think artificial delays sound very good. Rather
this is something the kernel should handle smoothly, imho.
As the one and only pseudoconstructive idea, for now, have you
tried running all the related tasks as 'nice'? Any help?


Igor