Re: (n-for-each-par-map 16 store download (reverse (iota (max-id))))) crash

[Amirouche Boubekki]

Hi Mark,

Thanks for your support and time.

On 2018-07-05 20:30, Mark H Weaver wrote:
> Hi,
>
> Amirouche Boubekki <address@hidden> writes:
>
> > I have a program that try to download hackernews locally.
> >
> > What it does is simple, it fetch the max identifier and
> > http-get each json value starting with the most recent
> > item. I use n-for-each-par-map with 16 threads I have
> > 8 cores.
> >
> > Here is the full program:
>
> [...]
>
> > (define (store pair)
> >   (if (null? pair)
> >       (format #t "X\n")
> >       (let ((port (open-file "hn.scm" "a")))
> >         (format #t "~a\n" (car pair))
>
> These calls to (format #t ...), which write to a port that is shared by
> multiple threads, should be performed while holding a mutex to prevent
> concurrent writes to the same port.

Ok, but it's called by:

  (n-for-each-par-map n sproc pproc lst1 lst2)

and the manual says about SPROC that :

  The sproc calls are made serially,
  in list element order, one at a time.

In my case SPROC is 'store', so my understanding is that
it should require no lock.

> I/O operations in Guile do not include built-in thread synchronization,
> at least not in the fast path cases.  However, an effort was made to
> avoid _crashes_ on common architectures in the event of concurrent use
> of the same port.  Our hope was that the worst that would typically
> happen is garbled I/O.  Perhaps we failed to realize that hope.

I saw that behavior while use fibers.

> > How can I debug this?
>
> It would be helpful to see a GDB backtrace from a crash in this 
> program.

I will retry without mutex and only gdb to reproduce the crash
and have a trace.

> Does it help to protect the shared port with a mutex?

I ran the program with gdb and a lock, it downloads more suff,
but it's also much slower.

>      Thanks,
>        Mark

Thanks again!

--
Amirouche ~ amz3 ~ http://www.hyperdev.fr