Re: [Help-gnunet] Corrupted sqlite db?

[Christian Grothoff]

On Sunday 09 March 2008, Milan wrote:
> Using 0.8.0pre0, everything was working well until today.  My computer
> did crash while gnunetd was running, and now I get this error (and only
> this one) as soon as my peer is connected:
>
> ERROR: `sqlite3_step' failed at sqlite.c:1341 with error: SQL logic
> error or missing database

That line is where the code issues an INSERT statement -- which happens 
without user action once other peers send data, which explains why it happens 
after the peer connects.

As far as the sqlite3-docs are concerned, this is an "internal error" (which I 
read as meaning "bug in sqlite3").  You could try to make sure that 
running "make check" (in sqstore_sqlite) passes (demonstrating that at least 
some basic GNUnet/sqlite functionality is working on your platform).  If that 
works, and if you want to pursue it faster, it might be a good idea to modify 
the test to use your "content.dat" file.  If it then crashes, you could file 
a more extensive report it to the sqlite developers and they might be able to 
reproduce & fix the issue.  

Now, the first thing I'd do is make sure that you run the latest stable sqlite 
version -- they may have fixed the bug already.

> I checked my /var/lib/gnunet/data/fs/content/content.dat file using:
> $ sqlite3 gnunet.dat .dump | sqlite3 content.dat.new
> $ mv content.dat.new content.dat
>
> And my database doesn't look to be corrupt.

I wonder if this load-dump is able to re-create the required indices as well.  
Now, without the indices, things should just be slow.

> Are there any checks I can perform? Is it possible that the database is
> consistent but that something is missing for gnunetd to be happy? There
> should definitely be a mechanism so that gnunetd repairs when possible
> the db.

Sounds to me like something in the database is corrupt that makes sqlite 
really unhappy.  I doubt that we can automatically repair it with the sqlite 
API.  I also don't know of any easy way to repair a broken sqlite database 
either (beyond what you've already tried).  This is one of the reasons why we 
have recommend MySQL for users who are able to handle the setup: the easier 
of recovery of crashed tables.  Now, obviously the (potential) data loss is 
not the issue here, but it may still be an interesting bug for the sqlite 
developers to resolve.  It certainly does not look like a bug in GNUnet 
itself.

> The file is "only" 200MB, so maybe I can send it if you need it.

I don't think *we* can do much with that file, but Richard Hipp or some other 
sqlite developer may find it useful.  

My 2 cents

Christian