Re: [Linphone-developers] Support packetization-mode=0 in H.264 When Using iOS VideoToolbox

[Li Qian]

Hi François,

Thank you very much for your clarification. I also notice the issue with setting MaxH264SliceBytes to VideoToolbox encoder session. I guess it is an iOS bug.

In the meantime, I'm still trying to use VideoToolbox for those call which the remote party do support packetization-mode=1, while using OpenH264 for those which don't. Here's the approach that I tried

1. Add "packetization-mode=1" when sending INVITE.

2. In the LinphoneCallStreamsRunning, I check whether the "send_fmtp" contains "packetization-mode=1" or not, which should indicate whether the remote party supports that or not.

3. I call  ms_factory_enable_filter_from_name(f, "MSOpenH264Enc", false); when it does not support.

However, the call continue to use the OpenH264 encoder for the call.

I also tried only call libmsopenh264_init when needed, but after calling that, future calls will never use VideoToolbox anymore.

Would you please suggest a correct way to change the encoder on-the-fly after receiving SIP OK?

Thank you in advance.

Looking forward to your reply.

Best Regards,

Li

On Mon, Nov 21, 2016 at 4:01 PM, François Grisez <address@hidden> wrote:

Hi Li,

 

Usually we set our H264 encoding filters to use the rfc3984 packer in 0 mode. But to do that we set the encoders to ensure they give us encoded data smaller or equal to the MTU. The VideoToolbox framework in theory enables us to set such a restriction through the MaxH264SliceBytes encoder property. But experience shows us this property does not work on devices we used for testing.

 

Given that we had to set the rfc3984 packer to 1 mode in order it fragments packets at RTP layer because IP fragmentation is not equally supported between all the Internet providers. Even some does not support it at all and drops packets that does not match MTU causing holes in the video stream.

 

Then, for all these reasons, we doesn't think that it is possible to have a good video quality if you set the rfc3984 packer to 0 mode. Maybe that could work on specific ISP network but not universally.

 

Best regards,

--

François Grisez

Software Engineer

Belledonne Communications

On Friday, November 18, 2016 4:25:01 PM CET Li Qian wrote:

Dear Linphone developers,

When using Linphone iOS app, with the iOS native H.264 encoder/decoder which uses VideoToolbox.framework, when call to some video phone model, it is one way video. The video phone cannot see the video, but Linphone iOS can see the video.

After hours of investigation, I found out that, when using VideoToolbox.framework to encode the video, it will call

rfc3984_set_mode(&ctx->packer_ctx, 1);

to tell the RTP packer to do fragmentation, i.e. packetization-mode=1, which results in FU-A packets. I compared the WireShark capture, I think those video phones which cannot see the video may not support that mode. So, I tried to do

rfc3984_set_mode(&ctx->packer_ctx, 0);

ctx->packer_ctx.maxsz = INT_MAX;

and hoping that IP layer can take care of the fragmentation if necessary. Then the video quality becomes very bad...

Is it possible to make the encoder to support packetization-mode=0 while still having the good performance when using VideoToolbox.framework?

Looking forward to your reply.

Best Regards,

Li

_______________________________________________
Linphone-developers mailing list
address@hidden
https://lists.nongnu.org/mailman/listinfo/linphone-developers