Pool of threads with one producer and several consumers

[Amirouche Boubekki]

Héllo guilers!

This is more like system programming question, but hopefully someone
will enlighten me.

I'd like to create a pool of thread with one producer and several
consumers. This needs comes from the need to make my database server
multithread. The flow of the program I imagine is the following:

1. The client application need to do a query against the database

2. The client open a connection against the server

3. The client send the query

4. The server's main thread accept a new connection

5. The server dispatch the connection to consumer threads

6. The consumer thread reads the query, execute it, returns the result
   and close the connection

The point I'm trying to solve is the point 5, when the server main
thread receive a new connection it must send the new connection to one
of the consumer.

My first idea was to create a queue protected by a mutex. The
protected queue will be used:

- by the mainthread to produce new connections for consumers

- by the consumers to retrieve connections

I translated this into Guile as follow:

```
(define (make-handler queue mutex init proc)
  (lambda () ;; thread thunk
    (init)  ;; thread initilisation
    (let loop ()  ;; thread loop
      (lock-mutex mutex)  ;; try to pick a connection
      (if (not (empty? queue))
        (let ((item (pop! queue)))  ;; pick a connection
          (unlock-mutex mutex)
          (proc item))  ;; process connection
        (unlock-mutex mutex))  ;; XXX: else, there is nothing in the 
queue.
                               ;;      In theory I must put a sleep 
here, right?
      (loop))))
```

AFAIU, sleep'ing is not a good thing to do. So I am investigating how to 
use
select or epoll instead. But i'm not sure how this should happen.

Any ideas?

NB: Right now, I don't plan to have persistent connections. What I
mean is that once a query is executed the connection to the client is
closed. If you have idea on how to use persistent connection, I am
also interested.


--
Amirouche ~ amz3 ~ http://www.hyperdev.fr