Re: problem with futures?

[Mikael Djurfeldt]

Andy,

I started implementing futures, but have not been doing any Guile
development since some time.

The basic mechanism is there, but the implementation is incomplete. 
The major omissions are associated with exactly the two problems you
bring up.  These known bugs are: the handling of the future cache
needs to be better, and, (even more important) the initialization of
the dynamic context for a new futures is currently not done at all.

The last point is a bug (the implementation is incomplete) which has
to be fixed before any release.  I've pointed this out before.

On 12/21/05, Andy Wingo <address@hidden> wrote:
> (define (nsum n)
>   (apply + (let ((x 1))
>              (par-map (lambda (y) (set! x (1+ x)) (+ y x))
>                       (make-list n 0)))))
>
> I get this:
>
> guile> (nsum 100)
> $1 = 5150
> guile> (nsum 100)
> $2 = 5150
> guile> (nsum 100)
> $3 = 5150
> guile> (nsum 100)
>
> Backtrace:
> In current input:
>    5: 0* [nsum 100]
>    1: 1  [apply #<primitive-generic +> ...
>    1: 2*  (let* ((x 1)) (par-map (lambda (y) (set! x #) (+ y x))
> (make-list n 0)))
>    1: 3   [par-map #<procedure #f (y)> (0 0 0 0 0 0 0 0 0 ...)]
> In /opt/guile/share/guile/1.7/ice-9/threads.scm:
>   52: 4   [map #<primitive-procedure future-ref> ...
>   53: 5*   [apply #<primitive-generic map> #<procedure #f args> ...]
> In unknown file:
>    ?: 6    [map]
>    ?: 7*   [map #<procedure #f args> (0 0 0 0 0 0 0 0 0 ...)]
>    ?: 8*   [#<procedure #f args> 0]
> In /opt/guile/share/guile/1.7/ice-9/threads.scm:
>   55: 9*   (future (apply mapper (proc . arglists)))
>
> /opt/guile/share/guile/1.7/ice-9/threads.scm:55:20: In expression
> (future (apply mapper #)):
> /opt/guile/share/guile/1.7/ice-9/threads.scm:55:20: Cannot allocate
> memory

This is probably simply that your OS/memory can't handle more than 100
pthreads (including the memory space for the stack of each thread).

> >From then on out I can't call nsum any more. Perhaps threads assigned to
> futures are not being gc'd properly.

Futures are cached for re-use.  This makes overhead of futures lighter
than that of ordinary threads.  Currently, there is no mechanism for
clearing the future cache.   As long as the application is using a
reasonable number of thread (relative to what the OS can cope with),
there is no big reason to clear it.  However, for robustness against
bugs and extreme use, there should be a set of rules for clearing it. 
For example, it could be cleared if no futures have been created
within the last 10 ms.  Also, one could have some upper limit on the
number of futures being cached.

For now, I will not work further on that code.  To the Guile
developers I suggest that you do what you want.  Fix it, or back out
the future feature entirely.

M