playing audio from an AVI file...

hiya xerxes,

I wonder what could be gained from examining my avi extracter code when the problem I am trying to solve is why an audio buffer doesn't work in joal when it works fine in java sound. I would say the audio buffer extraction is good.

the problem I have with joal is that it doesn't provide an "AudioFormat" equivalent so it is expecting the audio buffer to be different that what I am providing.

Please note that Sound3d is a simplified API on top of OpenAL
The javadoc for the sound3d Buffer configure is flawed since it do not mention all the possible formats that OpenAL support.
http://jogamp.org/deployment/jogamp-current/javadoc/joal/javadoc/com/jogamp/openal/sound3d/Buffer.html#configure%28java.nio.ByteBuffer,%20int,%20int%29

Internally sound3d Buffer uses OpenAL al.alBufferData to setup the buffer.

al.alBufferData supports the four standard formats mentioned in Buffer

static int     AL_FORMAT_MONO16
static int     AL_FORMAT_MONO8
static int     AL_FORMAT_STEREO16
static int     AL_FORMAT_STEREO8

+ all OpenAL extension formats thus the javadoc for Buffer forget to mention the possibility to use the following formats found in ALExtConstants!

static int     AL_FORMAT_51CHN_MULAW
static int     AL_FORMAT_51CHN16
static int     AL_FORMAT_51CHN32
static int     AL_FORMAT_51CHN8
static int     AL_FORMAT_61CHN_MULAW
static int     AL_FORMAT_61CHN16
static int     AL_FORMAT_61CHN32
static int     AL_FORMAT_61CHN8
static int     AL_FORMAT_71CHN_MULAW
static int     AL_FORMAT_71CHN16
static int     AL_FORMAT_71CHN32
static int     AL_FORMAT_71CHN8
static int     AL_FORMAT_IMA_ADPCM_MONO16_EXT
static int     AL_FORMAT_IMA_ADPCM_STEREO16_EXT
static int     AL_FORMAT_MONO_ALAW_EXT
static int     AL_FORMAT_MONO_DOUBLE_EXT
static int     AL_FORMAT_MONO_FLOAT32
static int     AL_FORMAT_MONO_IMA4
static int     AL_FORMAT_MONO_MULAW
static int     AL_FORMAT_MONO_MULAW_EXT
static int     AL_FORMAT_QUAD_MULAW
static int     AL_FORMAT_QUAD16
static int     AL_FORMAT_QUAD16_LOKI
static int     AL_FORMAT_QUAD32
static int     AL_FORMAT_QUAD8
static int     AL_FORMAT_QUAD8_LOKI
static int     AL_FORMAT_REAR_MULAW
static int     AL_FORMAT_REAR16
static int     AL_FORMAT_REAR32
static int     AL_FORMAT_REAR8
static int     AL_FORMAT_STEREO_ALAW_EXT
static int     AL_FORMAT_STEREO_DOUBLE_EXT
static int     AL_FORMAT_STEREO_FLOAT32
static int     AL_FORMAT_STEREO_IMA4
static int     AL_FORMAT_STEREO_MULAW
static int     AL_FORMAT_STEREO_MULAW_EXT
static int     AL_FORMAT_VORBIS_EXT
static int     AL_FORMAT_WAVE_EXT

Thus in oder to help you figure out which one of these OpenAL formats that are supported on your hardware JOAL and matches your sample buffer you may want to use the com.jogamp.openal.util.ALHelpers
http://jogamp.org/deployment/jogamp-current/javadoc/joal/javadoc/com/jogamp/openal/util/ALHelpers.html

static int     getALSampleType(int sampleSize, boolean signed, boolean fixedP)
Returns the AL sample type matching the given audio type attributes, or ALConstants.AL_NONE.

getALFormat(int alChannelLayout, int alSampleType, boolean hasSOFTBufferSamples, AL al, ALExt alExt)
Returns a compatible AL buffer format given the AL channel layout and AL sample type.

This class should provide you with the tools needed to find a matching AL format that you can pass to the Buffer.configure format integer argument.

You can get AL and ALExt needed for the ALHelpers class to do its runtime detection using ALFactory.getAL() and ALFactory.getALExt()

Cheers
Xerxes

On 03/17/2014 12:47 PM, TheLittleP [via jogamp] wrote:
> hiya sven,
> thanks for showing interest in my problem and the information you have provided.
>
> I have written my own avi demultiplexer and decoder. the video plays fine but
> its just that I cant seem to get JOAL to play the audio buffers. the audio
> plays fine in java sound but I would really like to take advantage of joal for
> the 3D effects.

Now I am getting impressed Sir!

- Do you offer your demultiplexer and decoder w/ a free software license ?
- What are it's capabilities (container formats, decoder formats .. ) ?
- Last but not least .. and we all hate this issue, how about the patents ?
  (Now I said it .. sorry)
  (We still figure out how to provide a decoder w/ JOGL .. w/ getting the
   lawyers involved.)

.. good stuff!

~Sven

sven,
I doubt there is anything original with my code lol. it is all gathered from various sources and I have hacked it to suit my needs. most code is written to be as widely as compatible as possible but my intention is to use 1 format for my project so I have ripped 95% of the guts out to simply give me a skinny version. you are welcome to use it however you see fit.

xerxes,
yeah, I have been down that route if you look at the 9th post on this thread. unfortunately the ALHelper returned the exact code for MONO 8 which of course didn't help at all.

I didn't realise that all the formats listed in ALExt are compatible with alBufferData. I will try going through every format listed to see if I get lucky.

many thanks

I am at a loose end now.

I have tried all the formats listed and none of them work.

with my testing I was selecting different formats, converting using javax.sound.JavaSoundRenderer and it doesn't seem to matter what format I changed the audio buffer to they all played fine using the java sound renderer. but none would play in joal.

I have absolutely no idea why joal wont work. joal plays the wav file fine. java sound plays the wav file. java sound plays the avi but joal wont.

You had forgotten to rewind the bytebuffer

in Audio.java use this and it will work with JOAL OpenAL:

public static Buffer[] getBuffers(byte[] _buffer)
{
  Buffer[] b=AudioSystem3D.generateBuffers(1);
  ByteBuffer bb=ByteBuffer.allocate(_buffer.length);
  bb.put(_buffer);
  bb.flip(); // dont forget to rewind the bytebuffer to the beginning!
  b[0].configure(bb,Buffer.FORMAT_MONO8,11024);
  return b;
}

cheers
Xerxes

fantastic!!! I could kiss you lol.

this has been perplexing me for a number of weeks now. I even went on a 2 week holiday to have a break and come back with a clear head. and I was starting to become despondent lol.

I only had the first second play but least it played. I guess only 1 second played due to how poorly my make shift player queues the buffers - I will have that fixed in a jiffy.

thank you so much!!!
I knew it would be something ridiculously obvious lol.

TheLittleP wrote

fantastic!!! I could kiss you lol.

this has been perplexing me for a number of weeks now. I even went on a 2 week holiday to have a break and come back with a clear head. and I was starting to become despondent lol.

I only had the first second play but least it played. I guess only 1 second played due to how poorly my make shift player queues the buffers - I will have that fixed in a jiffy.

thank you so much!!!
I knew it would be something ridiculously obvious lol.

Great to hear its working on your side as well!

While i looked through your code i noticed the simplicity of your a/v synchronization. I like the way you associated unpacked audio buffers with the unpacked video frame making a/v sync a snap by simply dumping the associated audio buffers to the sound-card when displaying the new video frame. Neat!

Cheers
Xerxes

Im quite taken aback that something I created is "neat" lol

There is an issue with it though. It currently waits the frame time (66milliseconds) and then fetches the audio and video buffers. The problem is it takes too long to process the audio buffer so the audio sink effectives runs out and stops and hence why only a second or 2 plays.

My solution has been to pre-fetch the buffers of the next frame and submit the audio buffer so it is processed and queued before the current buffer runs out. I then wait the 66ms before submitting the image buffer and it works a treat. I create a new thread for each buffer processing so the main thread with a wait of 66ms has minimal latency.

I have wondered if the memory overhead of creating 16 new threads every second would cause a problem but I presume as they are most likely added to the garbage collector before the next 1 is created that it shouldn't be too much of an issue.

I have been a right film director today with lights, camera and action lol. The performers I have to run the script with have been exceptional!

Many Thanks Xerxes

Regards
P

On 03/18/2014 06:31 PM, TheLittleP [via jogamp] wrote: Creating and starting a thread is _very_ expensive.

Maybe look how we do it in our
stream worker (GLMediaPlayerImpl.StreamWorker) - just keep one thread
open .. while working w/ the stream - that should do it.

Ofc .. you may spawn of more threads for parallel decoding ..
but even here, I would use a thread pool and 'wait a while' before
killing them - maybe a user just pressed pause and wants to continue
a millisecond later ..

Further more, you may want to see how we use and impl. the ringbuffer,
which works lock free and allows a zero copy queuing of A/V data.

>
> I have been a right film director today with lights, camera and action lol.
> The performers I have to run the script with have been exceptional!

Sounds like fun, great.

~Sven

thank you sven, I will definitely look into the suggestions you made.
I'm sure all the buffers can be processed efficiently and effective using 1 thread and I will look to see how this has been done before.
I just slapped this together to get the tech working.
many thanks