Re: [GNUnet-developers] Large disk quota

[Christian Grothoff]

On Thursday 01 April 2004 01:40, Jussi Eloranta wrote:
> Hi,
>
>
> If I want to insert large amount of data without indexing (e.g. insert
> -n) then what is the recommended way of doing it?

Use mysql (very highly recommended).  Use -r for recursive insertion, if you 
want to get directories add -b.  That should be all (in addition to -n).

> I tried the following. I set disk quota to 50 gb, so that there is
> enough space. After this (I did remember to reinit the database; gdbm)
> the gnunetd refused to startup (ARGH (at xmalloc.c:some number, which I
> lost - anyway it is the source line number I guess)).
> I simply increased the limit for the check (in xmalloc.c) if(size >
> 1024 * 1024 * 40) to if(size > 1024 * 1024 * 80) and after
> that it started up and seems to be functioning properly even after
> inserting huge amount of data (with -n). What is this check for?

I did not think there was any legitimate code that would allocate more than 40 
MB, so to protect against unknown overflow bugs (and allocations that might 
be problematic for machines with not so much memory), this check was put into 
place.

> Is it safe to make it larger?

Yes.  I'll try to see where that huge allocation came from (I suspect it's the 
bloomfilter), you're probably the first to try such a gigantic amount of 
space and I guess we should change the code to allow that :-).

Christian