Re: JOGL media player?
Posted by Sven Gothel on Apr 26, 2013; 2:12pm
URL: https://forum.jogamp.org/JOGL-media-player-tp4027810p4029051.html
On 04/26/2013 03:06 PM, Xerxes Rånby [via jogamp] wrote:
>
> I have checked the code and the current FFMPEGMediaPlayer back-end needs
> improvement.
Yes, very good review Xerxes!
Me, the designer of GLMediaPlayer :), stopped working
on the multithreaded feature after I finished the Android impl.
The latter uses multithreading, hence it's smooth w/o drop.
Indeed, a fill-thread should be added to decode the n texFrames
and soon the n audioFrames until full (-> Ringbuffer).
The getNext*() shall fetch the next data from the ringbuffer
(non blocking) or wait until one is available (blocking)
and push it to the sink (audio) or return it (video texure).
Indeed .. an audio packet would drop a frame ('even worse').
Great analysis.
Thx, Sven
>
> What happens are:
> at 60 frames/s the cube calls
> final TextureSequence.TextureFrame texFrame = texSeq.getNextTexture(gl, true);
> http://jogamp.org/git/?p=jogl.git;a=blob;f=src/test/com/jogamp/opengl/test/junit/jogl/demos/es2/TextureSequenceCubeES2.java;hb=HEAD#l357
>
> This call gets forwarded from GLMediaPlayerImpl and gets handled directly by
> FFMPEGMediaPlayer
> http://jogamp.org/git/?p=jogl.git;a=blob;f=src/jogl/classes/jogamp/opengl/util/av/GLMediaPlayerImpl.java#l149
> http://jogamp.org/git/?p=jogl.git;a=blob;f=src/jogl/classes/jogamp/opengl/util/av/impl/FFMPEGMediaPlayer.java;hb=HEAD#l374
> And here is the problem.. The designer of GLMediaPlayerImpl have allocated
> memory inside the texFrames in order to buffer decoded frames but the current
> implementation of FFMPEGMediaPlayer never touches this texFrames buffer
> instead it only return the latest decoded video frame in sequence.
>
> Even worse is that FFMPEGMediaPlayer only decode one frame using
> readNextPacket0 each time getNextTextureImpl is called and this will create
> pauses and stuttering because the packet may contain audio or subtitle frames
> thus a call to readNextPacket0 may not generate any new decoded video at all.
> Also we may miss some video frames since readNextPacket0 only return the first
> video frame decoded from the packed and a packet may contain more than one frame.
>
> So all in all, i expect playback to be slower than real-time because about
> half the packets inside a video stream is audio packets and for these packets
> no new video frame is returned when calling readNextPacket0 .. this is luckily
> compensated on some hardware that can update the display 60fps and only get
> new video frames every second time thus 30fps and most video formats expects a
> play rate at 29.9fps.. so its pure luck that we feel that the current
> implementation work.
>
URL: https://forum.jogamp.org/JOGL-media-player-tp4027810p4029051.html
On 04/26/2013 03:06 PM, Xerxes Rånby [via jogamp] wrote:
>
> I have checked the code and the current FFMPEGMediaPlayer back-end needs
> improvement.
Yes, very good review Xerxes!
Me, the designer of GLMediaPlayer :), stopped working
on the multithreaded feature after I finished the Android impl.
The latter uses multithreading, hence it's smooth w/o drop.
Indeed, a fill-thread should be added to decode the n texFrames
and soon the n audioFrames until full (-> Ringbuffer).
The getNext*() shall fetch the next data from the ringbuffer
(non blocking) or wait until one is available (blocking)
and push it to the sink (audio) or return it (video texure).
Indeed .. an audio packet would drop a frame ('even worse').
Great analysis.
Thx, Sven
>
> What happens are:
> at 60 frames/s the cube calls
> final TextureSequence.TextureFrame texFrame = texSeq.getNextTexture(gl, true);
> http://jogamp.org/git/?p=jogl.git;a=blob;f=src/test/com/jogamp/opengl/test/junit/jogl/demos/es2/TextureSequenceCubeES2.java;hb=HEAD#l357
>
> This call gets forwarded from GLMediaPlayerImpl and gets handled directly by
> FFMPEGMediaPlayer
> http://jogamp.org/git/?p=jogl.git;a=blob;f=src/jogl/classes/jogamp/opengl/util/av/GLMediaPlayerImpl.java#l149
> http://jogamp.org/git/?p=jogl.git;a=blob;f=src/jogl/classes/jogamp/opengl/util/av/impl/FFMPEGMediaPlayer.java;hb=HEAD#l374
> And here is the problem.. The designer of GLMediaPlayerImpl have allocated
> memory inside the texFrames in order to buffer decoded frames but the current
> implementation of FFMPEGMediaPlayer never touches this texFrames buffer
> instead it only return the latest decoded video frame in sequence.
>
> Even worse is that FFMPEGMediaPlayer only decode one frame using
> readNextPacket0 each time getNextTextureImpl is called and this will create
> pauses and stuttering because the packet may contain audio or subtitle frames
> thus a call to readNextPacket0 may not generate any new decoded video at all.
> Also we may miss some video frames since readNextPacket0 only return the first
> video frame decoded from the packed and a packet may contain more than one frame.
>
> So all in all, i expect playback to be slower than real-time because about
> half the packets inside a video stream is audio packets and for these packets
> no new video frame is returned when calling readNextPacket0 .. this is luckily
> compensated on some hardware that can update the display 60fps and only get
> new video frames every second time thus 30fps and most video formats expects a
> play rate at 29.9fps.. so its pure luck that we feel that the current
> implementation work.
>
signature.asc (911 bytes) | Free forum by Nabble | Edit this page |