Re: proposal for improvements in indexing

[Yngve Svendsen]

At 13:08 08.11.2001 -0800, Dirk Bergstrom wrote:
> whilst trying to solve the ongoing problems in PR 218, i've become 
> frustrated with the gnats index & database lock system.  it's crufty and 
> error prone, and it needs some serious work.
>
> what's really needed is to move to a better index format, like gdbm.  i 
> *think* that would allow us to get rid of the full database lock (for most 
> cases).  we would only need PR locks, and the gdbm_open() call (which, 
> presumably, does not allow two writers on the same dbm file).
>
> this would make the system a *lot* more robust.  it would improve speed 
> (many transactions would need to read only one PR entry, instead of the 
> whole index).  it would reduce the possibility of index corruption 
> (reading & writing thousands of index entries for each edit seems like a 
> bad idea).  it would reduce lock contention (two processes could 
> simultaneously edit different PRs).
>
> of course, that's real work.  but i think it should be considered for 4.1 
> or 4.2...

Isn't this one of the benfits we would gain from implementing support for 
proper relational databases as backends? I am not sure that would take much 
more work than rewriting the locking system, and we would also be likely to 
gain enormously in performance from doing that.

There was some discussion about implementing modular database backends in 
these mailing lists some time ago, but the thread died at some point.

Are there anyone who have experience implementing this kind of thing out there?

Yngve Svendsen
Gnatsweb maintainer