Re: Delimited continuations to the rescue of futures

[Mark H Weaver]

Hi Ludovic,

address@hidden (Ludovic Courtès) writes:
> Mark H Weaver <address@hidden> skribis:
>> In other words, if you have 4 cores and call 'par-map' on a list of 1000
>> elements, the main thread will only be used to process 1 out of 1000
>> elements, and only 3 cores will be used to process the other 999.
>> Is that right?
>
> No.  :-)
>
> That’s what would happen with current stable-2.0.  But in
> ‘wip-nested-futures’, the main thread behaves as a worker while waiting
> for the tail future to complete (which addresses your main concern, I
> think, as we had discussed on IRC.)  Thus, all cores are always used.

Ah, okay, good! :)

I guess the one remaining concern I have is that if there are any
long-running futures in the process, then any 'touch' could take a very
long time to complete, even if the future it is waiting for is a very
short job.

For example, (par-map - '(1 2 3)) could take several minutes to complete
if (par-map process-image list-of-images) is being done in another
thread.

This doesn't sit well with me.  It would be good if we could avoid it,
but at the very least we'll have to warn users about this in the
documentation.

I should mention that this problem already exists in Guile 2.0, and is
not related to your recent work.

   Thanks,
     Mark