Corrupted frame or transform Mac OSX (OK on win/linux)

Our project https://jaerproject.org has been using JOGL for about 10 years now very successfully and happily, but we always had a problem on Mac/OSX with rendering of the event camera output, where alternating frames were showing up as weirdly transformed with black background. Now finally we bought a mac mini to try to debug the problem but still cannot figure it out.  This follows with the wonderful release of JOGL2.5.0 that finally allowed us to update jaer to use Java 21 and netbeans 20 and make real install4j installers for it.

A rendered frame should look like this (here I zoomed out a lot to show the context better):
The scene is from a recording with DAVIS frame event camera of spinning coins



But every other frame looks like this on Mac OSX machine:



The problem is shown in the 2 videos in this folder: https://www.dropbox.com/scl/fo/6vhxmwqv85btlpfwcgqdo/h?rlkey=oald0p3wixny5y6z5o3q3m3l3&dl=0

These videos are screen capture videos of the behavior on OSX and Windows.
In each video I put breakpoints on the display() method and the internal JOGL swapBuffers calls.

I don't know if it some handing of the frame buffer objects in our code or the way we make textures or something else.
The code is really complicated for me to follow but I can fully appreciate Sven's comments about how handling the FBO is a PIA on OSX :-)

This folder also has the jogamp debug logs



If anyone has some idea about what is going on it would very helpful.

Hi Tobi,

interesting project, thank you.
In case your org has a budget, please consider <https://forum.jogamp.org/Call-for-funding-and-contractual-work-tp4042741p4043281.html> .. shameless, but a necessity.
Great that the 2.5.0 release was also helping your project.

So from my gut feelings this looks like a data-race condition with (as you also investigated)
with the buffer updates (swap, copy, ..). An occasional 'glFinish()' here-and-there
in your data-feed code may help locate the missing sync? Of course, this would need
to be properly addressed later on.

The background is not cleared and even the content is mangled plus duplication
and wrong transformation.

Aside from the root cause, I would attempt to use a simple GLAutoDrawable w/o FBO first,
i.e. GLWindow or our AWT GLCanvas.
GLJPanel perhaps with disabled GLSL shader: '-Djogl.gljpanel.noglsl=true'.

If anyhow possible, please extract a small application exposing the issue.
Having something more compact to read with less dependencies etc
surely lowers the bar to have a look at it.
You could also use synthetic texture data (assuming the objects are life fed) etc etc.

Yes, handling OSX is some work indeed as they change their API every other release.
Another current incompatibility seems to have appeared w/ 14.4 .. also on the pile
to find a sponsor - a big task in general to keep the project alive these days :-/

Hi Sven,Thanks for instantaneous response. Yes, we have a discretionary budget for some critical software support. Please write me to arrange a contract.

The JOGL coding in jAER was written while we were complete novices at OpenGL and also Java and so it shows. We are completely reliant on Swing for UI interaction and make heavy use of a variety of font rendering methods.  And yes, jAER uses a mole of dependencies but it is pretty easy to setup; I could do it in 1h on a brand new Mac Mini with zero mac experience in last 20 years; the only hassle was installing homebrew to install libusb.

I currently develop jAER via single dropbox on windows/linux/mac in netbeans and that is now I made the videos that I uploaded (with VNC to the mac mini) so it is pretty easy to compare and debug.

I can try to write a standalone project that exposes this bug but that will take me a few days. Otherwise I can ask you for a recommendation: Should I change from using top level GLCanvas to using top level GLWindow? Could that possibly affect this problem? I'm totally clueless about the inner working and don't even know what NEWT stands for or what is the role of jogamp and jogl. We began with GLCanvas and it works wonderfully for us except on Mac OSX.

One more thing: This problem only seems to appear in the 2d rendering pipeline: jAER's 3d rendering of the event sensor events doesn't seem affected. Our 2d rendering pipeline uses 2D textures to render the different camera outputs (event/frames/annotation). This suggests to me that the problem might be related to our texture handling code in https://github.com/SensorsINI/jaer/blob/master/src/net/sf/jaer/graphics/ChipRendererDisplayMethodRGBA.java

Tobi Delbruck wrote

Hi Sven,Thanks for instantaneous response. Yes, we have a discretionary budget for some critical software support. Please write me to arrange a contract.

Sounds great, will send you an email. Some hope left to keep things going :)

...
I currently develop jAER via single dropbox on windows/linux/mac in netbeans and that is now I made the videos that I uploaded (with VNC to the mac mini) so it is pretty easy to compare and debug.

Git is preferred, but using some side-channel for media exchange is always helpful ofc.

I can try to write a standalone project that exposes this bug but that will take me a few days.

Depending on the support outcome, I will dive into it as-is.

Otherwise I can ask you for a recommendation: Should I change from using top level GLCanvas to using top level GLWindow? Could that possibly affect this problem? I'm totally clueless about the inner working and don't even know what NEWT stands for or what is the role of jogamp and jogl. We began with GLCanvas and it works wonderfully for us except on Mac OSX.

In general I am not necessarily a friend of 'change things', i.e. leave a winning team.
However, depending on your UI requirements etc (the only reason for AWT/Swing) .. we might find
a path forward away from it. NEWT in general is more reliable so replacing the UI toolkit
could end up in a more modern and sane solution. That is the whole purpose of Graph/GraphUI,
our out-of-the-box UI solution, see https://jogamp.org/wiki/index.php?title=SW_Tracking_Report_Feature_Objectives_Overview#Graph
(yes, it is work-in-progress looking for funding etc etc, but I keep digging)

For this task here however, I would first like to fix it as-is
and leave the discussion to a future path forward open,
not pushing nor enforcing anything.

One more thing: This problem only seems to appear in the 2d rendering pipeline: jAER's 3d rendering of the event sensor events doesn't seem affected. Our 2d rendering pipeline uses 2D textures to render the different camera outputs (event/frames/annotation). This suggests to me that the problem might be related to our texture handling code in https://github.com/SensorsINI/jaer/blob/master/src/net/sf/jaer/graphics/ChipRendererDisplayMethodRGBA.java

That is very helpful, thank you!

Laters,

~Sven