JOGL media player?

I see a demo applet on the JogAmp site that looks like it should be displaying a cube with videos rendered on each of the faces.  The description of this applet refers to a JOGL media player.

For me, while the cube itself renders fine, each face appears to have a "test pattern" style image on it rather than a functioning video.

I have 2 questions about this:

1. Is there such a thing as the JOGL media player yet and what are its capabilities (support media types etc.)? The easiest way to answer this is probably to provide a link to where it is described.

2. If the answer to (1) is yes, why doesn't the applet actually display video content? I am running it on 64-bit Windows 7 using Java 7 Update 10 and NVIDIA GPU.

Thanks!

On 01/13/2013 09:21 PM, Qu0ll [via jogamp] wrote: GLMediaPlayer has a few implementations, see:
  <http://jogamp.org/git/?p=jogl.git;a=blob;f=src/jogl/classes/com/jogamp/opengl/util/av/GLMediaPlayer.java;h=3eca01986f49bbf72e73c75b55e447cec6491419;hb=224fab1b2c71464826594740022fdcbe278867dc#l51>

one available on all platforms is FFMpegPlayer:
 <http://jogamp.org/git/?p=jogl.git;a=blob;f=src/jogl/classes/jogamp/opengl/util/av/impl/FFMPEGMediaPlayer.java;h=4be2bcb587d6fbf70330fbf8f679f5fd7ee449ce;hb=224fab1b2c71464826594740022fdcbe278867dc#l52>

it is compatible w/ FFMpeg and libav  [0.6 ?, ] 0.7 and 0.8]
  <http://jogamp.org/git/?p=jogl.git;a=blob;f=src/jogl/classes/jogamp/opengl/util/av/impl/FFMPEGDynamicLibraryBundleInfo.java;h=32c8635532d32214d9ef006935875234eaaf98c2;hb=224fab1b2c71464826594740022fdcbe278867dc#l46>

On GNU/Linux distribution, usually the deployed version (if installed) shall work properly.

You can install ffmpeg/libav on Windows and OSX (precompiled versions, or by source)
and if available in PATH or system folder, it will be picked up
and play nicely - assuming one of the mentioned version is being used.
You can verify this in the video here:
  <http://www.youtube.com/watch?feature=player_detailpage&v=4gWStKCioi8#t=126s>

Currently only the Android impl. of our player supports audio,
but we will add JOAL audio output for the other implementations as well.
Volunteers ?

Adding support for native Windows/OSX implementations would be nice,
however - this hasn't been done yet.
Volunteers ?

~Sven

Precompiled ffmpeg bundles for Windows, OSX and Solaris:

  [Copy/Paste from FFMPEGMediaPlayer.java - for the impatient reader]:

  95  *   <li>Windows: http://ffmpeg.zeranoe.com/builds/</li>
  96  *   <li>MacOSX: http://www.ffmpegx.com/</li>
  97  *   <li>OpenIndiana/Solaris:<pre>
  98  *       pkg set-publisher -p http://pkg.openindiana.org/sfe-encumbered.
  99  *       pkt install pkg:/video/ffmpeg
 100  *       </pre></li>

Thanks very much for the info Sven.

I have installed FFmpeg and can now see the videos on the cube in the applet.

However, the videos play extremely slowly and the rendering is very jittery.  This is surprising because this machine is very powerful and has the current world's fastest graphics card (Nvidia GeForce 690 GTX).

Is this expected?  What could be the cause for this?  I have pondered on the following possibilities:

1. FFmpeg is inherently slow.
2. It's playing 6 videos at once so it has to be slow.
3. It's Java (slow).
4. It's something to do with the way the video frames are loaded into the texture.
5. It's not slow, it just runs slow on my machine.

Are any of these correct?  Any other ideas?

I would say 2. and 4.

One other thing which may be a factor is that Java performance inside a browser (i.e. applets) is said to be only 60-70% of what it is outside a browser (eg. desktop).  Not sure if this has anything to do with it.

On 01/15/2013 02:30 AM, Qu0ll [via jogamp] wrote:
> 2. It's playing 6 videos at once so it has to be slow.
Only one video -> texture.

> 3. It's Java (slow).
Nope.

> 4. It's something to do with the way the video frames are loaded into the
> texture.

Maybe due to the video loading itself, i.e. streaming from the internet.
The software decoded YUV frames are loaded into the texture and transformed to
RGBA w/ GLSL shader.

> 5. It's not slow, it just runs slow on my machine.
Well .. it's not slow on any device here.

Which stream do you use?
The orig. referenced low-res Big Buck Bunny?

>
> Are any of these correct?  Any other ideas?
Maybe try to download it .. and run it from the commandline
with passing the downloaded one.

~Sven

Qu0ll wrote

Thanks very much for the info Sven.

I have installed FFmpeg and can now see the videos on the cube in the applet.

However, the videos play extremely slowly and the rendering is very jittery.  This is surprising because this machine is very powerful and has the current world's fastest graphics card (Nvidia GeForce 690 GTX).

Is this expected?  What could be the cause for this?  I have pondered on the following possibilities:

...
4. It's something to do with the way the video frames are loaded into the texture.

I have checked the code and the current FFMPEGMediaPlayer back-end needs improvement.

What happens are:
at 60 frames/s the cube calls
final TextureSequence.TextureFrame texFrame = texSeq.getNextTexture(gl, true);
http://jogamp.org/git/?p=jogl.git;a=blob;f=src/test/com/jogamp/opengl/test/junit/jogl/demos/es2/TextureSequenceCubeES2.java;hb=HEAD#l357

This call gets forwarded from GLMediaPlayerImpl and gets handled directly by FFMPEGMediaPlayer
http://jogamp.org/git/?p=jogl.git;a=blob;f=src/jogl/classes/jogamp/opengl/util/av/GLMediaPlayerImpl.java#l149
http://jogamp.org/git/?p=jogl.git;a=blob;f=src/jogl/classes/jogamp/opengl/util/av/impl/FFMPEGMediaPlayer.java;hb=HEAD#l374
And here is the problem.. The designer of GLMediaPlayerImpl have allocated memory inside the texFrames in order to buffer decoded frames but the current implementation of FFMPEGMediaPlayer never touches this texFrames buffer instead it only return the latest decoded video frame in sequence.

Even worse is that FFMPEGMediaPlayer only decode one frame using readNextPacket0 each time getNextTextureImpl is called and this will create pauses and stuttering because the packet may contain audio or subtitle frames thus a call to readNextPacket0 may not generate any new decoded video at all. Also we may miss some video frames since readNextPacket0 only return the first video frame decoded from the packet and a packet may contain more than one frame.

So all in all, i expect playback to be slower than real-time because about half the packets inside a video stream is audio packets and for these packets no new video frame is returned when calling readNextPacket0 .. this is luckily compensated on some hardware that can update the display 60fps and only get new video frames every second time thus 30fps and most video formats expects a play rate at 29.9fps.. so its pure luck that we feel that the current implementation work.

On 04/26/2013 03:06 PM, Xerxes Rånby [via jogamp] wrote:
>
> I have checked the code and the current FFMPEGMediaPlayer back-end needs
> improvement.

Yes, very good review Xerxes!

Me, the designer of GLMediaPlayer :), stopped working
on the multithreaded feature after I finished the Android impl.
The latter uses multithreading, hence it's smooth w/o drop.

Indeed, a fill-thread should be added to decode the n texFrames
and soon the n audioFrames until full (-> Ringbuffer).

The getNext*() shall fetch the next data from the ringbuffer
(non blocking) or wait until one is available (blocking)
and push it to the sink (audio) or return it (video texure).

Indeed .. an audio packet would drop a frame ('even worse').

Great analysis.

Thx, Sven

Sven Gothel wrote

On 04/26/2013 03:06 PM, Xerxes Rånby [via jogamp] wrote:
>
> I have checked the code and the current FFMPEGMediaPlayer back-end needs
> improvement.

Yes, very good review Xerxes!

Me, the designer of GLMediaPlayer :), stopped working
on the multithreaded feature after I finished the Android impl.
The latter uses multithreading, hence it's smooth w/o drop.

Indeed, a fill-thread should be added to decode the n texFrames
and soon the n audioFrames until full (-> Ringbuffer).

The getNext*() shall fetch the next data from the ringbuffer
(non blocking) or wait until one is available (blocking)
and push it to the sink (audio) or return it (video texure).

Indeed .. an audio packet would drop a frame ('even worse').

Great analysis.

Thx, Sven

I am working on a JOGL branch with an updated JOGL FFMPEGMediaPlayer.
http://github.com/xranby/jogl/commits/FFMPEGMediaPlayer-audio

A quick fix for most of the stuttering on mobile devices using the FFMPEGMediaPlayer back-end is to keep on decoding packets until a video frame is available. It also make video playback in real-time on low end embedded machines using 30Hz screen refresh rate:
http://github.com/xranby/jogl/commit/9086ca1491c5fcf1ae318093b715045d9e30df87

Feel free to join the now ongoing forum and IRC discussion on how to implement a threaded low overhead 0-copy frame caching system with proper sync for audio and video to be used by the updated FFMPEGMediaPlayer back-end.

Cheers
Xerxes

On 04/28/2013 10:28 PM, Xerxes Rånby [via jogamp] wrote:
> I am working on a JOGL branch with an updated JOGL FFMPEGMediaPlayer.
> https://github.com/xranby/jogl/commits/FFMPEGMediaPlayer-audio
>
> A quick fix for most of the stuttering on mobile devices using the
> FFMPEGMediaPlayer backend is to keep on decoding packets untill a video frame
> is available. It also make video playback in real-time on low end embedded
> machines using 30Hz screen refresh rate:
> https://github.com/xranby/jogl/commit/9086ca1491c5fcf1ae318093b715045d9e30df87
>
I like this, looks promising to really decode all packet data!
  https://github.com/xranby/jogl/commit/48c7ccb60d045800960d0789a18c73619d8b58bd

Looking fwd to continue our a/v story with you this week - great stuff!

~Sven