Thread blocking issue with AWT (but not NEWT) on OSX

On 11/01/2012 02:06 AM, ac [via jogamp] wrote:
> Ok, I will open a bug report and try to identify the commit that introduced
> the deadlock.

Due to the high amount of changes happened (especially in OSX), it would help
a lot if you can name the last build (sha1, or URL) which works
and the first build which fails.

It must not be necessarily the very last and first, but .. you know,
it would make locating the root cause more easily.

As stated, the deadlock is related to OSX CALayer provoked
by using an Applet as a sub-component.
The deadlock appears regardless of FBO or PBuffer usage .. so yes,
it is a regression - since we had PBuffer support before.

Thank you.

~Sven

On 11/04/2012 07:35 PM, Sven Gothel wrote: I have validated the test case w/ numerous builds:
  - v2.0-rc10 (OK)
  - gluegen_584-joal_353-jogl_798-jocl_668
  - gluegen_586-joal_355-jogl_803-jocl_672
  - gluegen_599-joal_365-jogl_830-jocl_690
  - gluegen_600-joal_366-jogl_833-jocl_691
  - gluegen_603-joal_369-jogl_846-jocl_699
  - gluegen_603-joal_xxx-jogl_840-jocl_xxx
  - v2.0-rc11

Only RC10 seemed to work, some latter builds didn't freeze
but failed otherwise (no image etc).

Starting w/ jogl-build-833 (?), we offscreen CALayer worked properly
and the freeze happened.

This is not a regression, but highly probably a bug in the AWT implementation.

Please apply the workaround!

<http://jogamp.org/git/?p=jogl.git;a=commit;h=3a00361952120f8127a3014623de703ac696e036>

commit 3a00361952120f8127a3014623de703ac696e036
Author: Sven Gothel <[hidden email]>
Date:   Mon Nov 5 03:49:20 2012 +0100

    Analyzed OSX Deadlock w/ AWT Applet & CALayer: Andres Colubri's Test Case, Commit 77db6a5c22cb4a53cf911b4caf57127770c70968
   
    [1] - The AWT/Applet test case on OSX utilized offscreen CALayer, which is correct.
    [2] - the GLEventListener.display(..) method calls Frame.setTitle(".."); which is correct,
          but freezes when called right after GLEventListener.init(..)
   
    BUG on OSX/CALayer: If frame.setTitle() is issued right after initialization
    the call hangs in
       at apple.awt.CWindow._setTitle(Native Method)
       at apple.awt.CWindow.setTitle(CWindow.java:765) [1.6.0_37, build 1.6.0_37-b06-434-11M3909]
   
    Happens w/ Oracle's JRE7u9 as well!
   
    Workaround:
     - Set 'justInitialized' flag in GLEventListener.init(..)
     - Clear 'justInitialized' flag in GLEventListener.display(..) at the end
     - Skip Frame.setTitle(..) in GLEventListener.display(..) if 'true == justInitialized'
   
    Somebody may send a bugreport to Oracle / OpenJDK.

~Sven

Right, the change that brought the deadlock when calling Frame's methods is precisely the first commit of the FBObject, GLFBODrawable integration:

http://jogamp.org/git/?p=jogl.git;a=commit;h=4dd44b985fe0541be3a3bcd9045d201ed3ca2cc5

If I use the commit immediately before:

http://jogamp.org/git/?p=jogl.git;a=commit;h=9036376b7806a5fc61590bf49404eb71830de92f

with my test code, I actually get the following crash originating in the native libs, regardless of whether I use frame.setTitle() or not:

2012-11-06 17:39:32.306 java[36537:12003] invalid drawable
2012-11-06 17:39:32.309 java[36537:12003] -[NSView CGLPBufferObj]: unrecognized selector sent to instance 0x7fc5f2c998c0
2012-11-06 17:39:32.309 java[36537:12003] An uncaught exception was raised
2012-11-06 17:39:32.309 java[36537:12003] -[NSView CGLPBufferObj]: unrecognized selector sent to instance 0x7fc5f2c998c0
2012-11-06 17:39:32.310 java[36537:12003] (
        0   CoreFoundation                      0x00007fff8ada90a6 __exceptionPreprocess + 198
        1   libobjc.A.dylib                     0x00007fff8d8013f0 objc_exception_throw + 43
        2   CoreFoundation                      0x00007fff8ae3f6ea -[NSObject(NSObject) doesNotRecognizeSelector:] + 186
        3   CoreFoundation                      0x00007fff8ad975ce ___forwarding___ + 414
        4   CoreFoundation                      0x00007fff8ad973b8 _CF_forwarding_prep_0 + 232
        5   AppKit                              0x00007fff92a945be -[NSOpenGLContext setPixelBuffer:cubeMapFace:mipMapLevel:currentVirtualScreen:] + 59
        6   libjogl_desktop.jnilib              0x0000000112fe78cb setContextPBuffer + 203
        7   libjogl_desktop.jnilib              0x000000011304b28b Java_jogamp_opengl_macosx_cgl_CGL_setContextPBuffer__JJ + 43
        8   ???                                 0x0000000105d50eee 0x0 + 4392816366
)
2012-11-06 17:39:32.310 java[36537:12003] *** Terminating app due to uncaught exception 'NSInvalidArgumentException', reason: '-[NSView CGLPBufferObj]: unrecognized selector sent to instance 0x7fc5f2c998c0'
*** First throw call stack:
(
        0   CoreFoundation                      0x00007fff8ada90a6 __exceptionPreprocess + 198
        1   libobjc.A.dylib                     0x00007fff8d8013f0 objc_exception_throw + 43
        2   CoreFoundation                      0x00007fff8ae3f6ea -[NSObject(NSObject) doesNotRecognizeSelector:] + 186
        3   CoreFoundation                      0x00007fff8ad975ce ___forwarding___ + 414
        4   CoreFoundation                      0x00007fff8ad973b8 _CF_forwarding_prep_0 + 232
        5   AppKit                              0x00007fff92a945be -[NSOpenGLContext setPixelBuffer:cubeMapFace:mipMapLevel:currentVirtualScreen:] + 59
        6   libjogl_desktop.jnilib              0x0000000112fe78cb setContextPBuffer + 203
        7   libjogl_desktop.jnilib              0x000000011304b28b Java_jogamp_opengl_macosx_cgl_CGL_setContextPBuffer__JJ + 43
        8   ???                                 0x0000000105d50eee 0x0 + 4392816366
)
libc++abi.dylib: terminate called throwing an exception

Tested on OSX Mountain Lion 10.8.2 with a Radeon HD 6490M vcard.

On 11/06/2012 11:47 PM, ac [via jogamp] wrote: Yup, thats expected, since that bug is fixed in the upcoming commit.

However, culprit of the deadlock is the CWindow._setTitle(..)
implementation itself.

What I only can assume it that the Frame/Window setTitle impl.
somehow gets confused w/ the offscreen JAWT CALayer usage
and hence is a bug in that department.

~Sven

is the upcoming fix dealing specifically with the setTitle() call, or anything that is called from the frame object? I recall that doing frame.setResizeable() also resulted in the application hanging.

On 11/07/2012 03:09 AM, ac [via jogamp] wrote:
> is the upcoming fix dealing specifically with the setTitle() call, or anything
> that is called from the frame object? I recall that doing
> frame.setResizeable() also resulted in the application hanging.

Well, I cannot provide a fix to this I am afraid, hence there will be
no upcoming fix as long it's not 'our' fault within JOGL.

You can see in the git commit how I worked around it within
the 2 unit tests - sorry that's all so far.

We have pretty much isolated this issue by now (OSX, after CALayer creation),
but I cannot see us doing anything wrong here causing such a deadlock.

If setResizeable() fails to, I am afraid any mutable frame operation
will deadlock @ 1st init/reshape/display call right after context creation
(1st call).

I tried to remove all locking within our JAWTWindow of OSX,
and also checked locking in our CALayer etc .. but w/ no luck
I could see us blocking OSX's AWT forever.

Well while discussing this w/ you .. we may could workaround
by issuing one NOP display() for OSX to overcome this (?)

Maybe you can try this w/ the JOGL code (unit test, GLDrawableHelper)
if you feel confident.

BTW .. do we have the source code for the OSX Java7 AWT stuff ?
Is it the OpenJDK ? Maybe this will give us a clue whats going wrong.

Would be nice if we can analyze this issue a bit further together.

I know, it's far from being satisfied, but keep in mind the AWT/CALayer
code is new in OpenJDK/Oracle's JRE.

~Sven

On 11/07/2012 04:00 AM, Sven Gothel wrote: Further investigations shows that:
  - Current OpenJDK7 / jdk7u-dev doesn't use 'apple.awt.CWindow' anymore
  - It uses a new sun/lwawt/macosx/CPlatformWindow,
  - which utilizes a simple (sun/awt/AWTWindow.m)
       performSelectorOnMainThread:@selector(setTitle:)

I cannot see anything wrong here.

I also added an explicit [makeCurrent,release]/[makeCurrent,init,release]
in GLCanvas.setRealizedOnEDTAction(). This helps a bit already
but still deadlocks in like 1 of 8 runs.

Let's see what the latest OpenJDK7 OSX port behaves.

>
> ~Sven
>
>

Well, maybe I should just move to NEWT through NewtCanvasAWT. The deadlock doesn't appear to happen in that case.

On 11/08/2012 03:38 AM, ac [via jogamp] wrote:
> Well, maybe I should just move to NEWT through NewtCanvasAWT. The deadlock
> doesn't appear to happen in that case.
>

Yes, it's more stable - not just in regards to our code,
but the JRE OSX AWT implementation as well.

However, I will still complete the tests w/ jdk7u-dev
and try one more workaround - maybe I see something.

I am also sure that the upcoming OSX JRE/AWT release
will have these things stabilized.
At least we have source code (OpenJDK) to look at and analyze.

~Sven

On 11/08/2012 03:49 AM, Sven Gothel wrote: Good news, the setTitle, etc bug is not present w/ [openjdk] jdk7u!

On the other hand, I have found a threading bug in AWT GLCanvas
in case other threads injected 'display()' (animator) before
issuing removeNotify(). In that case, the display() is still
processed 'later on' after disposal.
Thanx to your test case, which triggers this situation occasionally.

After fix I will retest w/ jdk6.

> ~Sven

Great, thanks for the updates and fixes.

I have a more general question: would you recommend moving over to NEWT? Right now I implemented both paths (AWT/NEWT) in the Processing renderer, but keep using AWT mainly out of fear of compatibility issues... However, adding support for devices like the Raspberry Pi requires NEWT, as some tests by Xerxes Ranby seem to indicate.

Also, and as a purely speculative thought at this point: since JOGL fully supports the creation of FBO-supported offscreen surfaces, wouldn't be possible to create a purely headless OpenGL context, which then could be drawn into the native window? Wouldn't this solve all the windowing and event handling issues?

On 11/08/2012 06:09 PM, ac [via jogamp] wrote:
> Great, thanks for the updates and fixes.

Here is the fixing commit for OSX Java7 (should also fix behavior of other OS)
regarding GLAutoDrawable disposal w/o a created context - and JAWT locking w/o native peer:
   <http://jogamp.org/git/?p=jogl.git;&a=commit&h=d0f91a8ed17fbb1a7b56511c4e53a29e576f01af>

The unit tests contain a manual workaround, if required, see detection here:
  <http://jogamp.org/git/?p=jogl.git;a=blob;f=src/test/com/jogamp/opengl/test/junit/jogl/awt/TestGLCanvasAWTActionDeadlock01AWT.java;h=c51972c39b47afa9c1bf334d8127358666a5ed36;hb=d0f91a8ed17fbb1a7b56511c4e53a29e576f01af#l122>

>
> I have a more general question: would you recommend moving over to NEWT? Right
> now I implemented both paths (AWT/NEWT) in the Processing renderer, but keep
> using AWT mainly out of fear of compatibility issues... However, adding
> support for devices like the Raspberry Pi requires NEWT, as some tests by
> Xerxes Ranby seem to indicate.

Well, even though we try to be bug free even for native AWT / SWT,
we prefer NEWT due to it's more simple and straight forward implementation
and much less dependencies. Also the threading behavior is much better w/ NEWT,
even if used w/ AWT or SWT (-> NewtCanvasAWT and NewtCanvasSWT).
Also see: <http://jogamp.org/jogl/doc/NEWT-Overview.html>

Further more, there is no AWT/SWT on [headless] devices, only NEWT,
read Android and Raspberry Pi (currently).
[The Raspberry Pi is a special case though, and we may have JAWT support some day
when X11 EGL support is enabled]

However, if there is JOGL/AWT support, there is always NEWT support
but not the other way around :)

Hence in the long run to support most devices I would do:
  - Use NEWT incl. it's event listeners for basic implementation
  - Use NewtCanvas[AWT/SWT] if it must be added to such a UI

This migration path has been followed by professional and complex applications,
which were the actual driver of it (C3D CAD module).

The latter application runs well on Android now as well,
while being able to reuse the whole GL and NEWT module and stripping off AWT :)

>
> Also, and as a purely speculative thought at this point: since JOGL fully
> supports the creation of FBO-supported offscreen surfaces, wouldn't be
> possible to create a purely headless OpenGL context, which then could be drawn
> into the native window? Wouldn't this solve all the windowing and event
> handling issues?

Well, as you can see in the GLFBODrawableImpl, we still use a dummy window/surface handle
since a GL context is not complete otherwise. But that's not a real problem.
On EGL we use a pbuffer btw :)

Short answer, we now can utilize it - yes.
Now it's used for OSX CALayer.
- We could also use it for Raspberry Pi to complete AWT.
- We could also use this path for the Swing based GLJPanel.

Both cases would require us to composite the the offscreen pixels into the
used native components visible area, e.g. into AWT's canvas for example.

Otherwise many folks would like to still use plain onscreen rendering
via NEWT's GLWindow and AWT GLCanvas if supported.
The compositing windowmanager may force offscreen here as well,
but then they usually allow exclusive fullscreen mode without that!

Note that windowmanager on Windows and X11, compositing transparent surfaces is done
API transparent via their 'normal' API, unlike OSX w/ CALayer :)

So while tools and techniques are available and partially used
they are not the answer for everything.

The offscreen windowing handling incl. events etc, is actually more complicated
since all responsibility is on us here.
We have a bug in AWT->NEWT event translation for example .. and there are other issues.

So .. let's say it's quite complex - yes,
but we cannot force offscreen FBO and it's complexity to all our users,
especially not w/ NEWT and on mobile devices.

~Sven

On 11/08/2012 06:37 PM, Sven Gothel wrote:
~Sven

On 11/08/2012 06:09 PM, ac [via jogamp] wrote:
> Great, thanks for the updates and fixes.

Here is the fixing commit for OSX Java7 (should also fix behavior of other OS)
regarding GLAutoDrawable disposal w/o a created context - and JAWT locking w/o
native peer:

<http://jogamp.org/git/?p=jogl.git;&a=commit&h=d0f91a8ed17fbb1a7b56511c4e53a29e576f01af>

The unit tests contain a manual workaround, if required, see detection here:

<http://jogamp.org/git/?p=jogl.git;a=blob;f=src/test/com/jogamp/opengl/test/junit/jogl/awt/TestGLCanvasAWTActionDeadlock01AWT.java;h=c51972c39b47afa9c1bf334d8127358666a5ed36;hb=d0f91a8ed17fbb1a7b56511c4e53a29e576f01af#l122>

>
> I have a more general question: would you recommend moving over to NEWT? Right
> now I implemented both paths (AWT/NEWT) in the Processing renderer, but keep
> using AWT mainly out of fear of compatibility issues... However, adding
> support for devices like the Raspberry Pi requires NEWT, as some tests by
> Xerxes Ranby seem to indicate.

Well, even though we try to be bug free even for native AWT / SWT,
we prefer NEWT due to it's more simple and straight forward implementation
and much less dependencies. Also the threading behavior is much better w/ NEWT,
even if used w/ AWT or SWT (-> NewtCanvasAWT and NewtCanvasSWT).
Also see: <http://jogamp.org/jogl/doc/NEWT-Overview.html>

Further more, there is no AWT/SWT on [headless] devices, only NEWT,
read Android and Raspberry Pi (currently).
[The Raspberry Pi is a special case though, and we may have JAWT support some day
when X11 EGL support is enabled]

However, if there is JOGL/AWT support, there is always NEWT support
but not the other way around :)

Hence in the long run to support most devices I would do:
  - Use NEWT incl. it's event listeners for basic implementation
  - Use NewtCanvas[AWT/SWT] if it must be added to such a UI

This migration path has been followed by professional and complex applications,
which were the actual driver of it (C3D CAD module).

The latter application runs well on Android now as well,
while being able to reuse the whole GL and NEWT module and stripping off AWT :)

>
> Also, and as a purely speculative thought at this point: since JOGL fully
> supports the creation of FBO-supported offscreen surfaces, wouldn't be
> possible to create a purely headless OpenGL context, which then could be drawn
> into the native window? Wouldn't this solve all the windowing and event
> handling issues?

Well, as you can see in the GLFBODrawableImpl, we still use a dummy
window/surface handle
since a GL context is not complete otherwise. But that's not a real problem.
On EGL we use a pbuffer btw :)

Short answer, we now can utilize it - yes.
Now it's used for OSX CALayer.
- We could also use it for Raspberry Pi to complete AWT.
- We could also use this path for the Swing based GLJPanel.

Both cases would require us to composite the the offscreen pixels into the
used native components visible area, e.g. into AWT's canvas for example.

Otherwise many folks would like to still use plain onscreen rendering
via NEWT's GLWindow and AWT GLCanvas if supported.
The compositing windowmanager may force offscreen here as well,
but then they usually allow exclusive fullscreen mode without that!

Note that windowmanager on Windows and X11, compositing transparent surfaces
is done API transparent via their 'normal' API, unlike OSX w/ CALayer :)

So while tools and techniques are available and partially used
they are not the answer for everything.

The offscreen windowing handling incl. events etc, is actually more complicated
since all responsibility is on us here.
We have a bug in AWT->NEWT event translation for example .. and there are
other issues.

So .. let's say it's quite complex - yes, but we cannot force offscreen FBO
and it's complexity to all our users,
especially not w/ NEWT and on mobile devices.

~Sven