Re: [libmicrohttpd] Problem send large trunk of data over upgraded connection with epoll and https enabled

[Nguyen Xuan Viet]

Dear Christian,

Thanks for your quick fix and your explanation!
I tested the latest code on Git master and I see my problem is gone.
I'm waiting for next release to integrate it with my work.

Best regards,
Viet

On 9/28/20 3:37 AM, Christian Grothoff wrote:
> Dear Viet,
>
> I've now managed to fix the issue in Git master.
>
> Basically, we incorrectly decided that the read-buffer had been drained
> when gnutls_record_recv() returned a partial read. However, gnutls may
> return a partial read even if the read buffer has not been drained.
>
> Because we use edge polling, we must drain the buffer, which is only the
> case if gnutls_record_recv() returns 0.  I've fixed this and a few
> possibly related corner issues (with gnutls_record_uncork) in Git master.
>
> Happy hacking!
>
> Christian
>
> On 9/18/20 5:26 AM, Nguyen Xuan Viet wrote:
> > Dear Christian,
> >
> > I'm sorry but this bug still happen on latest version of this library.
> > The "test_upgrade_large_tls" still hangs with very large data send over
> > the upgraded connection.
> > We can reproduce this issue by increase "LARGE_STRING" and/or
> > "LARGE_REPLY_STRING" size to a huge string, for example duplicate
> > current "Hello..." string to 41 lines or more.
> >
> > This issue only happens with EPOLL + HTTPS, for other cases, it works
> > normally.
> > Could you please check this issue for me?
> >
> > Thanks,
> > Viet
> >
> > On 7/30/19 11:00 PM, Việt Nguyễn Xuân wrote:
> > > Dear Christian,
> > >
> > > Thanks for your reply.
> > > I'm very happy to hear your information. I will continue to work with your 
> > > latest source in the future.
> > >
> > > Best regards,
> > > Viet
> > > On 30/07/2019 01:04, Christian Grothoff wrote:
> > > > Dear Viet,
> > > >
> > > > Thanks for the additional details.  With your report, I was able to
> > > > reproduce the issue (test_upgrade_large.c added) and fix it in
> > > > bafd078984be5f846287107824eca99e6c6c42cc.
> > > >
> > > > Happy hacking!
> > > >
> > > > Christian
> > > >
> > > > On 6/24/19 7:16 PM, Việt Nguyễn Xuân wrote:
> > > > > Hi all,
> > > > >
> > > > > Sorry, I cannot reply your email, so I reply it here.
> > > > > I'm sorry, for the moment I cannot provide the code for you. But you can
> > > > > reproduce the issue by modify the src/microhttpd/test_upgrade.c, set the
> > > > > connection memory limit to a very low value, for example 512, this
> > > > > change should be made for HTTPS section. In the function "run_usock" and
> > > > > "run_usock_client", please replace the test string "Hello World" by a
> > > > > real larger string with length about 160 - 170 bytes. After that do a
> > > > > make check and you will see the test_upgrade_tls test hang forever.
> > > > >
> > > > > Thanks,
> > > > > Best regards,
> > > > > Viet
> > > > >
> > > > > ------------------------------------------------------------------------
> > > > > *From:* Việt Nguyễn Xuân
> > > > > *Sent:* Sunday, June 23, 2019 11:28 AM
> > > > > *To:* libmicrohttpd@gnu.org
> > > > > *Subject:* [libmicrohttpd]Problem send large trunk of data over upgraded
> > > > > connection with epoll and https enabled
> > > > >   
> > > > > Hi All,
> > > > >
> > > > > I'm using libmicrohttpd to implement a WebSocket server based on HTTP
> > > > > Upgrade connection.
> > > > > I'm using latest git revision of this library for my work.
> > > > > Everything works well for small message passes over upgraded connection,
> > > > > I tested with Chrome and Firefox browsers and they are all Ok.
> > > > > But I have a trouble when sending large trunk of message to the server,
> > > > > if the size of message is larger than 1/3 connection buffer size, it's
> > > > > very often that this message will be queued for next send or is not sent
> > > > > at all. For smaller messages, they will be sent and received immediately.
> > > > >
> > > > > This issue  only happens when I enable HTTPs combine with EPOLL internal
> > > > > thread, for other combination it seems that this issue does not happen.
> > > > > For example I can send large message normally with HTTPS + POLL internal
> > > > > thread or with HTTP + EPOLL internal thread.
> > > > >
> > > > > Could you please check this issue for me?
> > > > > Thanks a lot.
> > > > >
> > > > > Viet