Re: Fwd: Re: What is the scheduler about?

[Danny]

Maybe I should have been a little more exact/clear in my wordings.
I don't mean every http request is handled in its own thread, I mean
all code that is executed for one particular request happens in the
same thread that it is assigned to.

Anyway, using one GNUNET_PROGRAM_run on each thread sounds
interresting. However, I'm still scratching my head how I could let
gnunet and the web server communicate, even for just one thread.

Both the web server and gnunet would be running in their own loop, so
they have to be on different threads as far as I know. I can dispatch
from gnunet to the webserver. But I'd like to be able to do the same
the other way around honestly.


I've been looking at https://github.com/canndrew/gnunet-rs , and I
don't really like their code, but I guess rewriting the service APIs
would be the way to go then. Then I'm not really stuck using the built-
in scheduler and I can just use unix domain sockets and have all the
freedom the world about how I use those sockets. :)

It's going to be a bit more work I guess, but I guess it will be worth
it.

Groetjes,
Danny


On Sat, 2021-02-27 at 18:08 +0100, TheJackiMonster wrote:
> You should notice that even the calls of GNUNET_SCHEDULER_add_now()
> have to be on the same thread. So if you want to dispatch something
> from another thread you have to implement some custom IPC or
> synchronization first between your threads before calling any GNUnet
> API. ^^'
>
> However you can also startup an independent client/program using the
> GNUnet API on each thread. So you can still use the services running
> in
> the background without doing synchronization yourself. But it depends
> on the application your building and if full independance of your
> http-
> request-handling is wanted.
>
> By the way for optimization I wouldn't recommend starting a thread
> for
> each request but having a fixed size thread-pool which matches the
> hardware its running on.
>
> Jacki
>
> On Sat, 2021-02-27 at 16:43 +0000, Danny wrote:
> > Hey,
> >
> > Yeah so I was trying to run a webserver (for the UI) alongside
> > gnunet.
> > I made it so that gnunet starts on the main thread, and after it is
> > initialized, I spawn a thread for the http server. The http server
> > has
> > its own 'event/message loop', which handles each http request in
> > one
> > thread, but multiple workers exist that handle http requests.
> >
> > But essentially in my case each connection to a gnunet service is
> > utilized on only one tghread. I thought this would probably be safe
> > (enough).
> > But now that I think about it, I can see the issue if the API calls
> > try
> > to modify the same variables (like a message queue) that the
> > scheduler
> > uses (considering they access it from different threads).
> >
> > I guess I'll just have to dispatch all the api calls done on the
> > web
> > server thread(s) with GNUNET_SCHEDULER_add_now.
> >
> > Danny.
> >
> > On Sat, 2021-02-27 at 17:24 +0100, TheJackiMonster wrote:
> > > Hi,
> > >
> > > you shouldn't use the API with multiple threads because it is
> > > pretty
> > > much single-threaded. This may sound not great at first but
> > > having
> > > mutexes or something similar all over the place would make the
> > > whole
> > > framework quite complex.
> > >
> > > Also the client-side API communicates mostly through sockets with
> > > the
> > > services which are split into multiple separate processes. So you
> > > already get some kind of implicit synchronization by using these
> > > sockets and have some kind of parallelism with those multiple
> > > services.
> > >
> > > So if you want to develop an application using the GNUnet
> > > framework,
> > > I
> > > would suggest putting all calls regarding GNUnet into one thread
> > > and
> > > parallelize the other parts of the application. For example if
> > > you're
> > > developing a graphical application, you can fork the main process
> > > into
> > > two (one for the GNUnet API and one for the GUI).
> > >
> > > I did pretty much that for cadet-gtk because GTK3 is also mostly
> > > single-threaded. I also would not recommend using two much
> > > threads
> > > in
> > > a
> > > simple application because it can cause issues, you may never
> > > find.
> > >
> > > Happy hacking,
> > > Jacki
> > >
> > > On Sat, 2021-02-27 at 16:05 +0000, Danny via Mailinglist for
> > > GNUnet
> > > developers wrote:
> > > > Also, may I ask?
> > > > Is there any problem with using the API functions on other
> > > > threads
> > > > than
> > > > the one GNUNET_PROGRAM_run was called on?
> > > >
> > > > I am having weird segfaults/stack overlow errors at seemingly
> > > > random
> > > > places. I'm calling the *_connect (and afterwards *_disconnect)
> > > > functions, and then some API calls, all from another thread,
> > > > and
> > > > I'm
> > > > worried I have somehow caused unspecified behavoir or
> > > > something.
> > > > There is still no parallelism. I was thinking that since the
> > > > API
> > > > calls
> > > > simply  exchange process messages, it shouldn't matter from
> > > > which
> > > > thread I'm exchanging them. Or am I wrong here?
> > > >
> > > > Anyway,
> > > > Thanks a lot.
> > > > Danny
> >
> >
> >
> >