Hangs and Freezes (JOGL 2.2.4 & 2.3.0, MacOS X Yosemite, JDK 6, 7 & 8, SWT 3.8.2 & 4.4.2)

Hi all,

First of all, thanks to all the developers of JOGL. We've been relying on this library since 5 years for the GAMA platform (http://gama-platform.org) and, although it has been sometimes "tough" to deal with threading issues or rendering problems (since we combine AWT, SWT & JOGL in the same product), I musth say that I have been impressed by the robustness of JOGL. That is, until we decided (at last) to move from the old, deprecated JOGL 1.1 version to the "new" 2.x stream. We hadn't made the move before (despite several attempts last year) mainly because of human resources problems: finding someone skilled in OpenGL + AWT + Eclipse RCP was not completely evident and our resources are anyway limited.

Because we are bound to deliver a new version this summer, we decided to tackle this issue by ourselves now.

First of all, a word about the rendering subsystem of the platform: the general UI is based on Eclipse RCP (we havent made the move yet to the e4 stream, so we use Eclipse 3.8.2, although this may change in the following months), but, for performance reasons, we have a rendering based on Java2D (the "regular" displays, which allow small computer to run simulations) and/or OpenGL (using, thus, JOGL 1.1). Both displays rely on a somewhat "improved" version of the SWT_AWT bridge (another choice motivated by performances, as Draw2D is unbearably slow and the SWT GLCanvas was completely buggy at that time). Although highly acrobatic, this combination worked quite well for the last 4 versions of GAMA and our ~2000 users, who work on the 3 main OSes (MacOS X, Windows and Ubuntu).

Moving the OpenGL part of the rendering subsystem to the new JOGL libraries was not as painful as we thought (at least from a compilation point of view). The APIs have changed a bit, but nothing too complicated, and, soon, we had a "working" environment based on JOGL 2.2.4. Needless to say, it didn't work using the SWT_AWT bridge, and that is the first problem i want to report: without changing anything to our setup (beside the libraries and the adaptation to their API), all we obtained were gray displays or, somewhat randomly, working displays and gray ones side by side, but the working ones would freeze as soon as we resized a view. The logging of JOGL would reveal that the gray displays were considering the Canvas as not ready ("FPSAWTAnimator#00-Timer0: Info: GLCanvas display - skipped GL render, drawable not valid yet"), although they could be made "ready" by minimizing/maximizing the views several times in a row. The frozen ones, however, did not display anything, as if the animator had been deactivated somehow (no dispose message, nothing). A bit frustrating, to say the least.

Since JOGL has made tremendous progresses in supporting the SWT environment, we then decided to try out a pure SWT-based view for rendering OpenGL displays. I derived an "SWT" view based on the JOGL SWT GLCanvas, and tested the following combinations (only on MacOS X Yosemite, but we anyway need to have GAMA working on the 3 environments, on a 2011 17" MacBook Pro):

- JDK 1.6 (the latest from Apple), JOGL 2.2.4 & 2.3, SWT 3.8.2.
- JDK 1.7u76 (the latest 1.7 from Oracle), JOGL 2.2.4 & 2.3, SWT 3.8.2.
- JDK 1.8u25 (the latest 1.8 from Oracle), JOGL 2.2.4 & 2.4, SWT 3.8.2
- + all the above with a mixture of Eclipse 3.8.2 & SWT 4.4.2 (the latest from Eclipse).

Not all combinations worked flawlessly, but they did at least display something, which was already great news (no more fiddling with AWT/SWT inter threading issues !). However, they *all* showed the exact same problem, which is the main reason why I decided to write this (long) message:

1) The initialization of OpenGL is slow. The UI is blocked something like 1 or 2 seconds, while it is much faster using the AWT based views. I dont understand why, as this initialization has normally nothing to do with AWT or SWT. One reason could be that GLProfile.initSingeton() is called as soon as we open an OpenGL display, and maybe it's blocking the SWT thread and not the AWT one. I did not investigate more, and I could benchmark this, probably, by explicitly calling initSingleton() in a different thread, but this issue is not a priority right now.

2) Closing a view (either by hand or programmatically, when switching to a different model or simulation or relaunching a simulation) blocks the UI for 5 to 10 seconds, with the closing OpenGL viewpart being emptied, then shown during this period, until it is closed and another one is opened. The logs show in all cases the exact same messages: "java.lang.RuntimeException: Waited 5000ms for: …" after a call to lockSurface() by the animator thread + a stack trace corresponding to the various dispose() calls (from the ViewPane to the GLCanvas). I've tried to stop the animator first, do some preliminary dispose() of various components, destroy the context, etc. with no luck. Disposing a view containing a JOGL SWT GLCanvas always hangs (and this is a pity, since the view works quite well…). I just noticed that this behavior does not happen if we simply open a view, obtain the first displaying of the model (the animator is anyway running behind), do not run the simulation (i.e. no wiping, no replacement of the displayed objects occur) and then close it. As if having some "contents" in the view would prevent the view from closing.

3) Closing the application freezes it if an OpenGL part is still open. The same message is displayed, but then nothing happens and I have to force-quit it.

Pretty frustrating as I was quite close to the goal ! Now, I dont expect the members of the forum to magically solve this problem as it is clearly not a proper issue report. GAMA is a complex platform and it is perfectly possible that some of our design choices are the causes of this disappointing behavior.

Instead, I would like to have advices on how to test this progressively. In the light of what I have exposed, what kind of repeatable benchmark should I try to run to eliminate the different causes of this behavior. Are there some available in the test units of JOGL ? A simple setup with the JOGL SWT GLCanvas that I could try and then progressively complexify by myself ?

Also, does this report make sense for other users/developers of JOGL ? Sometimes, simply mentioning a similar problem can shed a light on a part of the software I hadn't thought about before.

Advices on how to proceed to obtain a working SWT-based OpenGL view are however what I expect the most, and I thank you in advance for your comment !

Cheers
Alexis

Hi

The maintenance of JOGL 1.1 was stopped several years ago, we won't be able to help you in the parts still relying on it. The SWT_AWT bridge is buggy too, I had to do something farfetched to make it work with the debug pipeline several years ago when I worked on an Eclipse RCP application but I admit that the SWT GLCanvas and NewtCanvasSWT aren't as stable as I want. As far as I remember, it's still possible to use JOGL 2 with the bridge but maybe something has been broken in a recent version, which I can't check as I no longer use SWT & Eclipse RCP. Feel free to fill a complete bug report, provide a tiny test case.

In my humble opinion, don't even try to support Apple JVMs. You should call GLProfile.initSingleton() earlier. You can find some unit tests in the Github repository:
https://github.com/sgothel/jogl/tree/master/src/test/com/jogamp/opengl/test

Good luck.

On 03/27/2015 03:29 AM, Alexis Drogoul [via jogamp] wrote:
> Hi all,
>
Thank you for your elaborate bug report.

>
> Since JOGL has made tremendous progresses in supporting the SWT environment,
> we then decided to try out a pure SWT-based view for rendering OpenGL
> displays. I derived an "SWT" view based on the JOGL SWT GLCanvas, and tested
> the following combinations (only on MacOS X Yosemite, but we anyway need to
> have GAMA working on the 3 environments, on a 2011 17" MacBook Pro):

I still have to update and test for 'MacOS X Yosemite' in general.
It should not cause issues .. but hey, they do change things
under the hood quite often.
As Julien suggested .. maybe perform it outside of SWT upfront,
maybe some blocking occurs and SWT waits a bit longer.
We do have SWT unit tests, of course.
Please check the jogl git repo.

Indeed it would be great, if you could extract a smallest possible
unit test for your (pure) SWT issues.

>
> Also, does this report make sense for other users/developers of JOGL ?

SWT is not my personal priority, however - there are users.
Hence .. it makes sense.

> Sometimes, simply mentioning a similar problem can shed a light on a part of
> the software I hadn't thought about before.
>
> Advices on how to proceed to obtain a working SWT-based OpenGL view are
> however what I expect the most, and I thank you in advance for your comment !

IMHO:
  - Check out bugzilla database, whether an already existing SWT bug exists
    and reply to it - or -

  - Create a new bug entry ..

  - Add unit tests via git repo ..

  - We should have a 'most important SWT' bug group
    (a new bug entry, all those most desired bugs will block)

The real problem is: Who will fix the bug?
While I have limited resources (SWT is low prio for me),
maybe somebody else likes to help you out?
At least .. I can assist.

In the end - pushing and managing SWT support by you
could help a lot!

Cheers, Sven

Jerome, Sven,

thanks a lot for your e-mails. They allowed me to clarify a bit the priorities and proceed (more or less) step by step. First of all, I definitely gave up on trying to use the SWT_AWT bridge again for this port. It seems that most of the threading problem were caused by the additions of OpenGL animator threads to the already complicated interaction between SWT and AWT threads. Too much for my small brain. Second, I took a look at several test units, and I think it gave me a good idea on how to correctly proceed — if I did not have a simulation platform to code, I would probably try to help you by giving a hand to the documentation, since the richness of the tests is underexploited in that respect.

Moving to SWT-only views simplified a lot of things, although I'm still struggling with awkward problems, but I will post them in separate threads in this forum. The good news is that, besides these (small) problems of refresh, interactions between views, resizing, everything works quite well with version 2.2.4 of JOGL. I've not yet made the move to the recent 2.3.1, mainly because textures were not working as expected on 2.3.0, and I wanted to tackle one problem at a time. I will do it later.

Regarding contributing a bit more efficiently to JOGL (beside documentation), it's true that I now have a number of case studies — and possibly some suggestions as well for the SWT port. Nothing formalized yet, but I will of course try to contribute.

Thanks again for your support — it has been excessively useful in this transition !

Cheers
Alexis

Which classes do you use to create textures? TextureIO? Which image file format?

Hi Julien

Sorry for being a bit on the slow side regarding answering the messages on this forum. I've now moved to a pure SWT implementation and  succeeded in solving most of the issues displayed earlier — except textures-related ones. Yes, we are using TextureIO, and the file format can be PNG, JPG but also internal image bitmaps (converted to BufferedImages).

Are there any special changes that we need to do to have textures working like they did on 1.1 ? I'd be happy to learn them, since it is now the main concern of our gradual move to JOGL 2.x; as I said, we dont see anything in 2.3, and we have a lot of issues in 2.2.x : flickering (I tried all the techniques displayed in the different threads), huge amounts of memory used (whereas it was ok in 1.1), and freezes when using a large number of textures.

I dont say that all these problems come from JOGL (and I'm sure they don't, actually); our management of textures is probably the cause here… Being able to address them one by one from the latest JOGL version would be then nice !  

Cheers
Alexis

Hi

Use a profiler to determine the root cause of the memory footprint increase, for example Eclipse MAT and/or JVisualVM.

JOGL 1 TextureIO = JOGL 2 AWTTextureIO (but keep in mind the latter might be removed in a later version).

Hi,

Yes, I figured out the various changes in JOGL2 compared to JOGL1. The problem is that the same code (nothing fancy, just creating a texture from an image, binding it to the current GL context, and displaying it) works flawlessly in JOGL 2.2.4, but does not display anything in JOGL 2.3.1. Any hint where it might come from ? I've tried to understand the changes between the two versions, but it's a bit complex to figure out.

Of course, it's perfectly possible that another part of our rendering system is the indirect cause of this malfunction, so I'll dig a bit more the changes introduced in JOGL 2.3.1.

Regarding the problems of memory, I guess it comes from a bad management of textures, but I still have to find the cause as well (the same code was working without these problems in JOGL1). Actually, we create and destroy a lot of textures (maybe a little bit too much, but that's another problem we will have to deal with later !) during the run of a simulation, and it seems that in some cases they are not properly destroyed…

Thanks anyway for your previous feedbacks — I'll come back to this forum if I make some progress — or not :) !

Cheers
Alexis

Hi

Actually, you have no guarantee about when OpenGL will effectively destroy your textures. Please provide a short self contained test and fill a bug report if you're really sure that the problem comes from JOGL itself.

Hi,

Just a small post to say that I actually managed (thanks to your various advices) to have a completely functional reimplementation of our rendering engine on SWT, with no AWT dependencies left. Bye-bye the broken and unreliable "AWT-SWT bridge" !

I discovered some interesting bugs in our own implementation by doing this conversion — this came as an unexpected bonus ! I have only two questions left:

1) One thing that bothered me for some time was that, when resizing views, visual glitches would appear. Setting the "swap interval" to zero (instead of 1 in the examples I had followed) solved this problem, but I wonder whether it is a general fix for SWT-based views or if it is specific to my own hardware — and in the latter case, how to determine the best settings for any given environment ?

2) The port is so clean that I've considered porting our (other) Java2D rendering subsystem to GLG2D, using a SWT canvas instead of an AWT one. Do you think it is easily feasible ? And what is the state of this library, as it doesn't seem to be maintained since 2013...

Thanks again for your feedback — I will more than likely come back to this forum regularly…

Cheers
Alexis

Hi

Alexis Drogoul wrote

Hi,

Just a small post to say that I actually managed (thanks to your various advices) to have a completely functional reimplementation of our rendering engine on SWT, with no AWT dependencies left. Bye-bye the broken and unreliable "AWT-SWT bridge" !

I discovered some interesting bugs in our own implementation by doing this conversion — this came as an unexpected bonus !

It's an excellent piece of news :)

Alexis Drogoul wrote

I have only two questions left:

1) One thing that bothered me for some time was that, when resizing views, visual glitches would appear. Setting the "swap interval" to zero (instead of 1 in the examples I had followed) solved this problem, but I wonder whether it is a general fix for SWT-based views or if it is specific to my own hardware — and in the latter case, how to determine the best settings for any given environment ?

Please look at our bug tracker, I wonder if there is already a bug report for this bug. Sometimes, the driver doesn't allow you to enable or disable the v-sync.

Alexis Drogoul wrote

2) The port is so clean that I've considered porting our (other) Java2D rendering subsystem to GLG2D, using a SWT canvas instead of an AWT one. Do you think it is easily feasible ? And what is the state of this library, as it doesn't seem to be maintained since 2013...

It's feasible but risky and not totally trivial. You should contain its maintainer of this library to know exactly what is going on.

Alexis Drogoul wrote

Thanks again for your feedback — I will more than likely come back to this forum regularly…

Cheers
Alexis

You're welcome.