Posted by Xerxes Rånby on Apr 26, 2013; 1:06pm
URL: https://forum.jogamp.org/JOGL-media-player-tp4027810p4029049.html

Qu0ll wrote

Thanks very much for the info Sven.

I have installed FFmpeg and can now see the videos on the cube in the applet.

However, the videos play extremely slowly and the rendering is very jittery.  This is surprising because this machine is very powerful and has the current world's fastest graphics card (Nvidia GeForce 690 GTX).

Is this expected?  What could be the cause for this?  I have pondered on the following possibilities:

...
4. It's something to do with the way the video frames are loaded into the texture.

I have checked the code and the current FFMPEGMediaPlayer back-end needs improvement.

What happens are:
at 60 frames/s the cube calls
final TextureSequence.TextureFrame texFrame = texSeq.getNextTexture(gl, true);
http://jogamp.org/git/?p=jogl.git;a=blob;f=src/test/com/jogamp/opengl/test/junit/jogl/demos/es2/TextureSequenceCubeES2.java;hb=HEAD#l357

This call gets forwarded from GLMediaPlayerImpl and gets handled directly by FFMPEGMediaPlayer
http://jogamp.org/git/?p=jogl.git;a=blob;f=src/jogl/classes/jogamp/opengl/util/av/GLMediaPlayerImpl.java#l149
http://jogamp.org/git/?p=jogl.git;a=blob;f=src/jogl/classes/jogamp/opengl/util/av/impl/FFMPEGMediaPlayer.java;hb=HEAD#l374
And here is the problem.. The designer of GLMediaPlayerImpl have allocated memory inside the texFrames in order to buffer decoded frames but the current implementation of FFMPEGMediaPlayer never touches this texFrames buffer instead it only return the latest decoded video frame in sequence.

Even worse is that FFMPEGMediaPlayer only decode one frame using readNextPacket0 each time getNextTextureImpl is called and this will create pauses and stuttering because the packet may contain audio or subtitle frames thus a call to readNextPacket0 may not generate any new decoded video at all. Also we may miss some video frames since readNextPacket0 only return the first video frame decoded from the packet and a packet may contain more than one frame.

So all in all, i expect playback to be slower than real-time because about half the packets inside a video stream is audio packets and for these packets no new video frame is returned when calling readNextPacket0 .. this is luckily compensated on some hardware that can update the display 60fps and only get new video frames every second time thus 30fps and most video formats expects a play rate at 29.9fps.. so its pure luck that we feel that the current implementation work.