no sound with jogamp on Arch Linux Raspberry

Hey guys,

first of all thank you for this huge project, which will, at some point I hope, help me write a little music server on my Raspberry.
Fora few days now I can not get simple things to work. I am running Arch on PI and installed latest version of jogamp, openAl, ffmpeg, and some more stuff.

This is what I achieved so far:

GLMediaplayerFactory creates an FFMPEGMediaPlayer object.
Add an EventListener.
Initialize stream.
Initialize GL.
Play an mp3 file.
...but nothing happens. Playing normally with ffplay works fine. Let me show you some code excerpts from my class MediaPlayer:

public class MediaPlayer implements GLMediaEventListener
{
        static GLMediaPlayer player;
       
        public static void main(String[] args)
        {
                MediaPlayer mediaPlayer = new MediaPlayer();
               
                player = GLMediaPlayerFactory.createDefault();
                player.addEventListener(mediaPlayer);
               
                if (player==null) {
                        System.out.println("Failed to create player!");
                } else {
                        System.out.println("Created new player: " + player.getClass().getName()); // prints ffpmegmediaplayer

                        resource = MediaPlayer.class.getResource(args[0]).getPath();

                        URI uri;

                        try
                        {
                                uri = new URI(resource);
                                player.initStream(uri,GLMediaPlayer.STREAM_ID_NONE, GLMediaPlayer.STREAM_ID_AUTO, GLMediaPlayer.TEXTURE_COUNT_DEFAULT);
                        }
                        catch (Exception e1)
                        {
                                e1.printStackTrace();
                        }

                        ...
                        ...
                        ...

                        // at some point the program ends
                        if (player.getState() == GLMediaPlayer.State.Playing) {
                                System.out.println("destroying player...");
                                player.destroy(null);
                        }
                }
        }

        @Override
        public void attributesChanged(GLMediaPlayer mp, int event_mask, long when)
        {
                System.out.println("\n***\nEvent mask changed: " + event_mask);
                System.out.println("Timestamp: "+ when);
                System.out.println("State of player: " + player.getState().toString() +"\n");
                               
                if ((event_mask & GLMediaEventListener.EVENT_CHANGE_INIT) !=0) {
                        try
                        {
                                System.out.println("player.initGL()...");
                                mp.initGL(null);
                        }
                        catch (IllegalStateException e)
                        {
                                e.printStackTrace();
                        }
                        catch (GLException e)
                        {
                                e.printStackTrace();
                        }
                        catch (StreamException e)
                        {
                                e.printStackTrace();
                        }
                }else if ((event_mask & GLMediaEventListener.EVENT_CHANGE_PAUSE) !=0) {
                        System.out.println("player.play()...");
                        System.out.println("mp.setPlaySpeed(1f) returned: " + mp.setPlaySpeed(1f)); //just to make sure
                        mp.play();
                }else if ((event_mask & GLMediaEventListener.EVENT_CHANGE_PLAY) !=0) {
                        System.out.println("playing...");
                        System.out.println(mp.toString()); // info output
                        System.out.println(mp.getAudioSink().toString()); // more info output
                }
        }
}


The program seems to run OK, but there is no sound. The AudioSink object shows always an invalid time stamp (getPTS()) equal to AudioFrame.INVALID_PTS and AudioSink.isPlaying shows false. Otherwise the objects look fine after issuing the play command in FFMPEG:

GLMediaPlayer[Playing, vSCR 2147483647, frames[p 0, d 0, t 0 (303.255 s)], speed 1.0, 204376 bps, Texture[count 0, free 0, dec 0, tagt 0xde1, ifmt 0xffffffff, fmt 0xffffffff, type 0xffffffff], Video[id -2, <unknown>, 0x0, glOrient false, 0.0 fps, 0.0 fdur, 0 bps], Audio[id 0, <Audio: mp3, 44100 Hz, stereo, s16p, 203 kb/s>, 203995 bps, 0 frames], uri /home/media/test.mp3]

ALAudioSink[init true, playRequested true, device OpenAL Soft, ctx 0x1cd35c5, alSource 1, chosen AudioDataFormat[sampleRate 44100, sampleSize 16, channelCount 2, signed true, fixedP true, packed, little-endian], al[chan Stereo, type s16, fmt 0x1103, soft true], playSpeed 1.0, buffers[total 20, avail 20, queued[0, apts -2147483648, 0 ms, 0 bytes], queue[g 20, l 39]

with -jogl.debug=true, after issuing the mp.play() method:

setPlaySpeed(1.0): Paused, 1.0 -> 1.0, GLMediaPlayer[Paused, vSCR 2147483647, frames[p 0, d 0, t 0 (303.255 s)], speed 1.0, 204376 bps, Texture[count 0, free 0, dec 0, tagt 0xde1, ifmt 0xffffffff, fmt 0xffffffff, type 0xffffffff], Video[id -2, <unknown>, 0x0, glOrient false, 0.0 fps, 0.0 fdur, 0 bps], Audio[id 0, <Audio: mp3, 44100 Hz, stereo, s16p, 203 kb/s>, 203995 bps, 0 frames], uri /home/klubi/media/PearlJamYellowLedbetter.mp3]

Thread-2: ALAudioSink: PLAY playImpl false, ALAudioSink[init true, playRequested true, device OpenAL Soft, ctx 0x1cd35c5, alSource 1, chosen AudioDataFormat[sampleRate 44100, sampleSize 16, channelCount 2, signed true, fixedP true, packed, little-endian], al[chan Stereo, type s16, fmt 0x1103, soft true], playSpeed 1.0, buffers[total 20, avail 20, queued[0, apts -2147483648, 0 ms, 0 bytes], queue[g 20, l 39]

Play: Paused -> Playing, GLMediaPlayer[Playing, vSCR 2147483647, frames[p 0, d 0, t 0 (303.255 s)], speed 1.0, 204376 bps, Texture[count 0, free 0, dec 0, tagt 0xde1, ifmt 0xffffffff, fmt 0xffffffff, type 0xffffffff], Video[id -2, <unknown>, 0x0, glOrient false, 0.0 fps, 0.0 fdur, 0 bps], Audio[id 0, <Audio: mp3, 44100 Hz, stereo, s16p, 203 kb/s>, 203995 bps, 0 frames], uri /home/media/testr.mp3]

Does anyone have any clue, where the problem could be? It seems to me, as if some native library is causing the problem.
Any hints are very welcome. Thanks.

klubi

please add a link to a git repository with the full code. its hard to spot what is wrong unless i can test the compete code.

Do the precompiled jogl-test.jar  MovieSimple or MovieCube examples work for you?
Example usage of JogAmp GLMediaPlayer using one of the examples found inside the jogl-test.jar
java -cp joal.jar:jogl-test.jar:gluegen-rt.jar:jogl-all.jar com.jogamp.opengl.test.junit.jogl.demos.es2.av.MovieCube -url file:///home/pi/test.mp3

More examples on how to setup and use JogAmp from the command line on the pi can be found at:
http://gist.github.com/xranby/55de4fd65bfadae26203

Some additional Raspberry Pi JOAL test examples can be found at:
https://gist.github.com/xranby/4160432

On 11/27/2013 10:47 PM, Xerxes Rånby [via jogamp] wrote: Please set the debug properties as well, since they may show you whats missing:
  -Djogl.debug=all -Djoal.debug=all

or more detailed for GLMediaPlayer and JOAL/OpenAL
  -Djogl.debug.GLMediaPlayer -Djogl.debug.GLMediaPlayer.Native -Djogl.debug.AudioSink -Djoal.debug=all


> More examples on how to setup and use JogAmp from the command line on the pi
> can be found at:
> http://gist.github.com/xranby/55de4fd65bfadae26203
>
> Some additional Raspberry Pi JOAL test examples can be found at:
> https://gist.github.com/xranby/4160432
>

~Sven

Hey guys,
thanks for the replies. Well I tried the jogl-test and it works. The difference in my code was supposed to be completely without a video stream. Only audio.

My idea was to:
create a default GLMediaPlayer,
initStream wit no video
then initGL(null)
then play()

I put up my first Git Repository:
https://github.com/dudewithasock/raspimusic/tree/master/raspberry/src

Basically the MediaPlayer class is the only important at this moment. Please let me know if I forgot something.
At the moment, this is a part of the current console output (without debug - that would be an overkill to post here):

*********************
player.play()...
mp.setPlaySpeed(1f) returned: true
SEEK XXX: Paused, frames[(p 0, d 0) / 0, 303.255 s], speed 1.0, dAV -2147483648, vSCR 2147483647, vpts 0, dSCR[-2147483647, avrg 0], aSCR 2147483647, apts -2147483648 ( 1 ), AudioSink[frames [p 0, q 0, f 20, c 20], time 0, bytes 0], Texture[count 0, free 0, dec 0]

***
Event mask changed: 4
Timestamp: 1386030150890
State of player: Playing

playing...
GLMediaPlayer[Playing, vSCR 2147483647, frames[p 0, d 0, t 0 (303.255 s)], speed 1.0, 204376 bps, Texture[count 0, free 0, dec 0, tagt 0xde1, ifmt 0xffffffff, fmt 0xffffffff, type 0xffffffff], Video[id -2, <unknown>, 0x0, glOrient false, 0.0 fps, 0.0 fdur, 0 bps], Audio[id 0, <Audio: mp3, 44100 Hz, stereo, s16p, 203 kb/s>, 203995 bps, 0 frames], uri /home/klubi/media/PearlJamYellowLedbetter.mp3]
ALAudioSink[init true, playRequested true, device OpenAL Soft, ctx 0x1de635a, alSource 1, chosen AudioDataFormat[sampleRate 44100, sampleSize 16, channelCount 2, signed true, fixedP true, packed, little-endian], al[chan Stereo, type s16, fmt 0x1103, soft true], playSpeed 1.0, buffers[total 20, avail 20, queued[0, apts -2147483648, 0 ms, 0 bytes], queue[g 20, l 39]
*********************


Thank again, I really appreciate your help.
klubi

Hi Xerxes,

thanks for the answer. I was almost expecting something similar ;)
If you said before, you want to use my class I would have done a better job writing nice code :) I think it is a good idea and there are surely lot of people interested in audio only playback. Feel free to use and modify, rename (whatever) the class as you will. Just make and use a real copy, since I don't know how long I'm going to keep the github repo in this form.
Now that I know where the problem is, I will also look more closely and try to find a nice solution when I find some time. This might require a modification to the FFMpegMediaPlayer class. I will let you posted.
Thanks, you've been great help.

Cheers,
klubi

Maybe it would be possible to get inspired by the FFMpegMediaPlayer class and write a simple class using the FFMpegs native method getnextpacket (I think that's the name of the method) and then somehow push the audio to AudioSink. Basically this would be a modification to the StreamWorker thread, just getting rid of all GL overhead in case of audio only streams.
No idea for when the Jogamp 2.1.3 is planned :) I'm kinda eager to make this work.

klubi wrote

Maybe it would be possible to get inspired by the FFMpegMediaPlayer class and write a simple class using the FFMpegs native method getnextpacket (I think that's the name of the method) and then somehow push the audio to AudioSink. Basically this would be a modification to the StreamWorker thread, just getting rid of all GL overhead in case of audio only streams.
No idea for when the Jogamp 2.1.3 is planned :) I'm kinda eager to make this work.

Stream Worker do not initialize GL when video is disabled using -vid -2 thus there should be no need to create a new "audio only" backend.

Sven Gothel have implemented the missing fuctionality and fixed bug 918
https://jogamp.org/bugzilla/show_bug.cgi?id=918
audio only playback now work using the GLMediaPlayer!
KUDOS!

2.1.3 is about to get released
(09:03:29) sgothel: http://jogamp.org/deployment/archive/master/gluegen_756-joal_506-jogl_1175-jocl_889/ <- will be 2.1.3 later today .. have to go now

An unsigned 2.1.3 "rc" test builds is available at: http://jogamp.org/deployment/archive/master/gluegen_756-joal_506-jogl_1175-jocl_889/archive/

Cheers
Xerxes

If you want to improve performance further on the raspberry pi then help implement raspberry pi support inside the OpenMAX backend, this backend was working back in 2008 on an early Nvidia phone devboard http://www.youtube.com/watch?v=D6Lkw3eZK1w - JOGL on APX 2500 (Tegra1) Dec 2008 . By using OpenMAX we can move both mp3 and mp4 decode from the CPU to the GPU on the Pi.

Example openmax code in /opt/vc/ on the pi.

JOGL OpenMAX GLMediaPlayer backend code is located at:
src/jogl/classes/com/jogamp/opengl/util/av/GLMediaPlayerFactory.java
src/jogl/classes/jogamp/opengl/util/av/impl/OMXGLMediaPlayer.java
src/jogl/native/openmax/jogamp_opengl_util_av_impl_OMXGLMediaPlayer.c
src/jogl/native/openmax/omx_tool.h
src/jogl/native/openmax/omx_tool.c

klubi wrote

Maybe it would be possible to get inspired by the FFMpegMediaPlayer class and write a simple class using the FFMpegs native method getnextpacket (I think that's the name of the method) and then somehow push the audio to AudioSink. Basically this would be a modification to the StreamWorker thread, just getting rid of all GL overhead in case of audio only streams.
No idea for when the Jogamp 2.1.3 is planned :) I'm kinda eager to make this work.

2.1.3 is released i updated your source and added some test script
https://github.com/xranby/raspimusic/commit/460fdad99e316917065a97927c68d4dd64e819dd

While testing on the raspberry pi i agree the GLMediaPlayerFactory.getDefault() took quite some time to initialize 15s on the Pi vs 0.6 seconds on a regular x86 desktop system. I will look into it and check if FFMPEGMediaPlayer can skip the GL initialization, that happens during its constructor, for audio only streams.

Cheers
Xerxes

Xerxes Rånby wrote

klubi wrote

Maybe it would be possible to get inspired by the FFMpegMediaPlayer class and write a simple class using the FFMpegs native method getnextpacket (I think that's the name of the method) and then somehow push the audio to AudioSink. Basically this would be a modification to the StreamWorker thread, just getting rid of all GL overhead in case of audio only streams.
No idea for when the Jogamp 2.1.3 is planned :) I'm kinda eager to make this work.

2.1.3 is released i updated your source and added some test script
https://github.com/xranby/raspimusic/commit/460fdad99e316917065a97927c68d4dd64e819dd

While testing on the raspberry pi i agree the GLMediaPlayerFactory.getDefault() took quite some time to initialize 15s on the Pi vs 0.6 seconds on a regular x86 desktop system. I will look into it and check if FFMPEGMediaPlayer can skip the GL initialization, that happens during its constructor, for audio only streams.

Cheers
Xerxes

MediaPlayer: Fix early main exit race-condition at startup.
https://github.com/xranby/raspimusic/commit/1b596352dc162e4ea1489f7c4ffa5d1af3c32eb9

The FFMPEGMediaPlayer depend on the OpenGL initialization in order for its native code to work thus desktop or mobile gl initialization has to be done and cant be skipped.
Its is possible to reduce the start-up delay substantially by using the jogl-all-mobile.jar instead of jogl-all.jar!
The mobile version skips detection of desktop OpenGL implementations; this alone makes the Pi startup ~5s quicker!

Cheers
Xerxes

I also already tried the 2.1.3 and it worked right away. Also using the jogl-no-awt took 16 seconds on my Raspi as well.
Will try the jogl-all-mobile version.
Thanks for the commits, I will merge them into my initial master. I just did not think this would go so far, that's why I did not bother with additional stuff like file-checking etc... :) Man, I wish I had more time!

Best regards,
Martin