Hello developers,
This is a new blog post about minimizing the latency in realtime video communication, If you also want to learn about how to speed up the player startup duration, please see our previous post (Minimizing the initial delay)
In audio/video communication, the latency should be as low as it can, in our previous blog post, We minimized the initial connecting delay. Although this seems to solve the latency (if We start fast, then We can catch the realtime, huh ?), it will not. Because, ffmpeg does not let you do this as ffmpeg has inner buffers and it’s too late when you request a packet by using av_read_frame method in application layer.
FFmpeg has some methods that let you seek in streams, for example, you can seek in a video file stream by a given time, like jumping from the start point which is 0 to the end time, how about the seeking in inner buffers ? Is this possible for realtime streams ? Please note that live streams has not duration property!
Fortunately YES, ffmpeg also lets you seek in by bytes, How can we get the size of buffers that ffmpeg keeps inside ? In this situation, avio_size method rescues us. This method can say the current size of buffers in bytes. Then, We need everything to tell the ffmpeg that please goto end in the buffers.
We’ve implemented an objective-c method to do above easily, the method’s name is seekInDecoderBufferByValue. The API docs about this method is here
How to use this objective-c method ?
It’s easy, We need to call this method when the stream is ready to play. So,
Firstly, open VKPlayerController.m file, and add a BOOL property to VKPlayerController class to avoid the method call more than once. For example, add “_callOnce” in VKPlayerController interface as below
@interface VKPlayerController ()
...
...
BOOL _callOnce;
}
Then, please find below lines in – (void)decoderStateChanged:(VKDecoderState)state errorCode:(VKError)errCode method in same file, and add the extra lines as below,
else if (state == kVKDecoderStatePlaying) {
...
_labelStatusEmbedded.hidden = YES;
_labelElapsedTimeEmbedded.hidden = NO;
[self showControlPanel:YES willExpire:YES];
_snapshotReadyToGet = YES;
if(!_callOnce) {//add this line
[_decodeManager seekInDecoderBufferByValue:0.5];//add this line
_callOnce = YES;//add this line
}
}
The parameter of this method can be between 0.0 and 1.0, to reduce the latency maximum, this parameter can be set to 1.0
Now, let’s look at the result video that We attached here,
As you see it’s not perfect but it’s really acceptable 🙂
In conclusion,
Do here and also here (Minimizing the initial delay) to minimize the latency for realtime communications
That’s all for this post now, in our next blog post, We have a plan to discuss the new features that coming with the 1.10 version such as, VKDECODER_OPT_KEY_PASS_THROUGH, GIF support , etc.
Have fun!