OSGi with Bndtools and Apache Felix throws exception

Hello,

I am trying to use JOGL within an OSGI application, that is using bndtools and Apache Felix.
The application seems to find all the native libraries from "jogl-all-natives-windows-i586.jar" (when using a 32-bit VM), but it crashes when trying to display the GLCanvas.
To isolate the issue, I set up the example from the tutorial: http://jogamp.org/wiki/index.php/Using_JOGL_in_AWT_SWT_and_Swing#JOGL_in_AWT

Without OSGI, the example runs alright, but as soon as I make an OSGI bundle from it, I get the following exception when trying to run it: (in this stacktrace, I tried using an Eclipse OSGI-Runtime Environment, but it didn't help.)

java.lang.ArrayIndexOutOfBoundsException: -1
        at jogamp.opengl.windows.wgl.awt.WindowsAWTWGLGraphicsConfigurationFactory.chooseGraphicsConfigurationImpl(WindowsAWTWGLGraphicsConfigurationFactory.java:169)
        at javax.media.nativewindow.GraphicsConfigurationFactory.chooseGraphicsConfiguration(GraphicsConfigurationFactory.java:420)
        at javax.media.opengl.awt.GLCanvas.chooseGraphicsConfiguration(GLCanvas.java:1181)
        at javax.media.opengl.awt.GLCanvas.addNotify(GLCanvas.java:571)
        at java.awt.Container.addNotify(Container.java:2769)
        at java.awt.Window.addNotify(Window.java:770)
        at java.awt.Frame.addNotify(Frame.java:487)
        at java.awt.Window.show(Window.java:1031)
        at java.awt.Component.show(Component.java:1651)
        at java.awt.Component.setVisible(Component.java:1603)
        at java.awt.Window.setVisible(Window.java:1014)
        at org.example.OneTriangularAWT.main(OneTriangularAWT.java:52)
        at org.example.Activator.start(Activator.java:14)
        at org.eclipse.osgi.framework.internal.core.BundleContextImpl$1.run(BundleContextImpl.java:711)
        at java.security.AccessController.doPrivileged(Native Method)
        at org.eclipse.osgi.framework.internal.core.BundleContextImpl.startActivator(BundleContextImpl.java:702)
        at org.eclipse.osgi.framework.internal.core.BundleContextImpl.start(BundleContextImpl.java:683)
        at org.eclipse.osgi.framework.internal.core.BundleHost.startWorker(BundleHost.java:381)
        at org.eclipse.osgi.framework.internal.core.AbstractBundle.start(AbstractBundle.java:299)
        at aQute.launcher.Launcher.update(Launcher.java:307)
        at aQute.launcher.Launcher.activate(Launcher.java:235)
        at aQute.launcher.Launcher.run(Launcher.java:139)
        at aQute.launcher.Launcher.main(Launcher.java:74)

For testing and to reduce possible causes, I've set this up as a single bundle, including the JOGL libraries and native libraries.

I hope someone can help me out, and I thank you in advance

I have successfully made JOGL into an OSGi bundle using the native JARs, though I haven't had the time to write up a tutorial yet. If you look at my GitHub projects

git://github.com/WadeWalker/com.jogamp.jogl.git
git://github.com/WadeWalker/name.wadewalker.OneTriangle.git

the first one shows JOGL made into an Eclipse plugin (with native JARs from all 5 platforms, without using fragments), and the second one shows how to use it in an Eclipse RCP plugin project. Maybe this is similar enough to what you're doing to be helpful?

Thank you for the quick response, unfortunately it produces the same error for me in bnd.
 I've copied your project and built a bnd.bnd for it.
bnd.bnd

Apparently it cannot load the native-jars like that, so I just decompressed the dlls for i586 from the JARs and placed them in the according folder. Now it finds the native libraries again and also once again throws the ArrayIndexOutOfBoundsException.

Your bundle settings don't look the same as my MANIFEST.MF. For example, you have gluegen-rt-natives-windows-i586.jar and jogl-all-natives-windows-i586.jar in the classpath, but they shouldn't be -- they don't contain Java classes, they're just archives containing .dlls. They should be in the same directory as your other JARs, though.

Also, you need to have a pretty recent version of JOGL to use the bundle resolver (as shown at the top of https://github.com/WadeWalker/name.wadewalker.OneTriangle/blob/master/src/name/wadewalker/onetriangle/OneTriangleView.java). This is needed for JOGL to be able to find the native JARs, due to the fact that OSGi uses custom classloaders. I'm using recent nightly builds.

Thank you, I think that may be it. I overlooked the bundle resolver in your example, because I just thought it was something that Eclipse needed.

About those native jars in the classpath: I didn't include them there initially, but when JOGL couldn't locate the libraries I tried to fix it, and apparently I didn't limit myself to logical approaches. I forgot to take that out in the bnd that I uploaded, sorry.

I don't have much time right now and can't test it, but I will do so tomorrow.
I'll post my results then,
Thank you very much

I've tried to implement my own JarUtil.Resolver, following your example. Unfortunately, there is no FileLocator in Felix, so I had to work around that. After parsing the URL (e.g "bundle://3.0:1/com/jogamp/common/os/Platform.class") myself and fetching the reference for the file, I noticed that this doesn't seem to be the information that JOGL wants. (it checks, whether the result starts with "jar:", which it does not: "bundle://3.0:0/com/jogamp/common/os/Platform.class")

I rewrote the path (in a quite ugly way, but enough for testing), so that it returns "jar:bundle://3.0:0/gluegen-rt.jar!..." and "jar:bundle://3.0:0/jogl-all.jar!..." respectively. Now it finds the correct libraries within the JARs.

Unfortunately, the same ArrayIndexOutOfBoundsException as before. Here is some log output, including some of my own debug messages

16:39:50.188 [main] DEBUG org.example.Activator - Starting Bundle
16:39:50.208 [main] DEBUG org.example.Activator - unresolved: bundle://3.0:1/com/jogamp/common/os/Platform.class
16:39:50.208 [main] DEBUG org.example.Activator - resolved: jar:bundle://3.0:0/gluegen-rt.jar!/com/jogamp/common/os/Platform.class
16:39:50.298 [main] DEBUG org.example.Activator - unresolved: bundle://3.0:2/jogamp/nativewindow/NWJNILibLoader.class
16:39:50.298 [main] DEBUG org.example.Activator - resolved: jar:bundle://3.0:0/jogl-all.jar!/jogamp/nativewindow/NWJNILibLoader.class
java.lang.ArrayIndexOutOfBoundsException: -1
        at jogamp.opengl.windows.wgl.awt.WindowsAWTWGLGraphicsConfigurationFactory.chooseGraphicsConfigurationImpl(WindowsAWTWGLGraphicsConfigurationFactory.java:169)
        at javax.media.nativewindow.GraphicsConfigurationFactory.chooseGraphicsConfiguration(GraphicsConfigurationFactory.java:420)
        at javax.media.opengl.awt.GLCanvas.chooseGraphicsConfiguration(GLCanvas.java:1181)
        at javax.media.opengl.awt.GLCanvas.addNotify(GLCanvas.java:550)
        at java.awt.Container.addNotify(Container.java:2769)
        at java.awt.Window.addNotify(Window.java:770)
        at java.awt.Frame.addNotify(Frame.java:487)
        at java.awt.Window.show(Window.java:1031)
        at java.awt.Component.show(Component.java:1651)
        at java.awt.Component.setVisible(Component.java:1603)
        at java.awt.Window.setVisible(Window.java:1014)
        at org.example.OneTriangularAWT.main(OneTriangularAWT.java:52)
        at org.example.Activator.start(Activator.java:94)
        at org.apache.felix.framework.util.SecureAction.startActivator(SecureAction.java:645)
        at org.apache.felix.framework.Felix.activateBundle(Felix.java:1977)
        at org.apache.felix.framework.Felix.startBundle(Felix.java:1895)
        at org.apache.felix.framework.BundleImpl.start(BundleImpl.java:944)
        at aQute.launcher.Launcher.update(Launcher.java:307)
        at aQute.launcher.Launcher.activate(Launcher.java:235)
        at aQute.launcher.Launcher.run(Launcher.java:139)
        at aQute.launcher.Launcher.main(Launcher.java:74)

I don't have the DLLs any where any more except in the native-JARs, so I'm quite sure it uses those now.
Maybe you have another idea, what goes wrong.

Here's what my resolver does:

unresolved: bundleresource://28.fwk1822163802/com/jogamp/common/os/Platform.class
resolved: jar:file:/D:/Users/Wade/Documents/Blog/jogl2/com.jogamp.jogl/gluegen-rt.jar!/com/jogamp/common/os/Platform.class
unresolved: bundleresource://28.fwk1822163802:1/jogamp/nativewindow/NWJNILibLoader.class
resolved: jar:file:/D:/Users/Wade/Documents/Blog/jogl2/com.jogamp.jogl/jogl-all.jar!/jogamp/nativewindow/NWJNILibLoader.class

So I'm not sure yours is correct -- it should be doing something like this:

unresolved: bundle://3.0:1/com/jogamp/common/os/Platform.class
resolved: jar:file:/C:/actual/path/to/file/gluegen-rt.jar!/com/jogamp/common/os/Platform.class
unresolved: bundle://3.0:2/jogamp/nativewindow/NWJNILibLoader.class
resolved: jar:file:/C:/actual/path/to/file/jogl-all.jar!/jogamp/nativewindow/NWJNILibLoader.class

JOGL uses the java.net.URL class to open the JAR file, then unpack its contents (the DLLs) into a temporary directory somewhere, so it needs to be able to find the JAR file with an absolute path on your file system. The URL class doesn't recognize the bundle: protocol (that's something OSGi has a custom classloader for), hence the need for a resolver.

You might also try running my name.wadewalker.OneTriangle project (which requires my com.jogamp.jogl project), just to make sure that JOGL works on your system and there's not some other odd problem going on.

I tried it with absolute paths first, but with the same result.
I actually manually build the URL, because I didn't any automatic way to make Felix build it for me.
I believe that it's able to find and load the native libraries correctly, because it throws an exception otherwise and of course it did so frequently until I resolved that.

I was not able to actually run your name.wadewalker.OneTriangle project, because there was some weird SWT ClassNotFoundException, that I didn't have the time or need to resolve yet. I know that JOGL works on my system because I can run the Tutorial without OSGI, and it displays correctly.

Have you tried enabling JOGL's debug logging to see if anything shows up there? If you add "-Djogamp.debug=all -Dnativewindow.debug=all -Djogl.debug=all -Dnewt.debug=all" to your Java arguments, it should produce a lot more debug output.

Since JOGL works outside of OSGi for you, there must be some Java runtime changes that OSGi does which JOGL just doesn't like, maybe the debug output will give a pointer to that.

Thank you for the suggestion and sorry for taking up so much of your time.
Unfortunately I don't know how the correct log should look like (except for the stack traces).

I'd be very thankful if you could take a look at it. http://pastebin.com/K8kLDvMK

Maybe you can see what goes wrong.

I can also discard the possibility, that it's just one machine that doesn't work.
I tried it on two different machines (Win7 and Win8, both 64 bit), with several different VMs (1.6 and 1.7, both 32 and 64 bit)

Unfortunately due to other JNI dependencies I'm restricted to a 32-bit VM, so my preferred runtime is 1.7 32-bit. (for now)
But during those tests, I built a small separate workspace, so that there won't be any interference.

Thank you again.

Looking at the log, I see

wglARBPFIDs2GLCapabilities: Invalid pfdID 0/1: 0
updateGraphicsConfigurationARB: wglARBPFIDs2GLCapabilities failed with 1 pfd ids

It's failing to choose a pixel format for some reason. I'm not sure why that might happen, maybe Sven could help on this if you submit a bug report at https://jogamp.org/bugzilla/.

One other thing you could try: find a Felix-based program that shows some normal AWT graphics window, and look to see how/where they create their window. Then put your JOGL initialization stuff in the same spot in your program. I noticed you're trying to start JOGL in the Activator.start() method, but this may be too early in the app lifecycle, or on a non-GUI thread, or something else that won't let JOGL work properly.

In my Eclipse RCP apps, I don't start JOGL until the app is creating the window that I'm going to display into (inside View.createPartControl()) -- maybe there's an equivalent location for Felix apps?

I actually took the AWT example from the Wiki, which works alright.
To create the OSGi Test bundle, I just added the BundleActivator and called the existing main-method of the example from the activate method.

I don't know if that causes a problem, but in my main application I have a fully working modular AWT GUI, that works up until the point when I try to display the GL canvas (which throws the same ArrayIndexOutOfBoundsException)
I'm sure, I followed common practice there and of course have a dedicated thread for the GUI there.

I'm going to try your idea the day after tomorrow.

As you suggested, I'm going to file a bug report now.
Thank you again for your time and effort.

This is actually quite embarrassing... while collecting information for the bug report, I noticed that the log was apparently automatically truncated by Eclipse, so quite a bit of the top was missing (it's now 1.8 MB big). Sorry about that.

But I think I found the issue here, but first, here is the log (separated into two parts)

The first one is the log output of: GLProfile.getDefault();
ArrayIndexOutOfBoundsException-with-osgi-felix-and-awt.txt

The second one happens, when the application tries to display the window (and the GLCanvas with it)
ArrayIndexOutOfBoundsException-with-osgi-felix-and-awt---part-2.txt
(I don't know, why I used pastebin for the earlier one)

Apparently the bundle cannot find/access "sun.java2d.opengl.OGLUtilities" which is included in the JRE.
I already tried including the package as an import and/or as a system package within the manifest-file, but it didn't help.
I think I have to set "org.osgi.framework.system.packages.extra" correctly, so that the bundle is allowed access.

I think I'm already a bit too tired to figure out why it didn't work yet, but I think that may be it. Please correct me if I'm wrong.
Otherwise, I'm going to try the day after tomorrow (or maybe even tomorrow, if I'm able to find the time).

I also did NOT file a bug report... because it doesn't seem to be one.

Here's what the correct log looks like when I run my OneTriangle Eclipse RCP app: correct_log.txt

You may be right, that OGLUtilities class seems to be present in my classpath. When I run, instead of ClassNotFoundExceptions, I get this in the log:

OGLUtilities.UNDEFINED = 0
OGLUtilities.WINDOW = 1
OGLUtilities.PBUFFER = 2
OGLUtilities.TEXTURE = 3
OGLUtilities.FLIP_BACKBUFFER = 4
OGLUtilities.FBOBJECT = 5

This thing kept bothering me, so I kept trying.

Apparently Felix couldn't find the OGLUtilities even if it was within the build path (and class). Apparently JOGL uses a Class.forName(...) to get the class, which is not recommended practice for OSGI. Interestingly while trying to access the same class (also with Class.forName(...)) from within the bundle itself (actually from the test-bundle), I got the class just fine... and apparently the subsequent call within JOGL also succeeds then... I consider that a temporary workaround, not a fix.
While writing this post, I noticed another interesting point. When I removed my own Class.forName(...) again, JOGLs own Class.forName(...) still finds it. I'm a bit confused, because I didn't change anything else (except for rebuilding and stepping through the stack a few times).

But for now, it helps me to get to the next Exception... which is again the ArrayIndexOutOfBoundsException while trying to chooseGraphicsConfigurationImpl(WindowsAWTWGLGraphicsConfigurationFactory.java:169)

I sifted to the log, but it is just very massive if you don't know what you're looking for.

So I kindly request, that you take a look at it again.
ArrayIndexOutOfBoundsException-with-osgi-felix-and-awt-2013-03-26-0844.txt

sneakpaw wrote

Apparently JOGL uses a Class.forName(...) to get the class, which is not recommended practice for OSGI. Interestingly while trying to access the same class (also with Class.forName(...)) from within the bundle itself (actually from the test-bundle), I got the class just fine... and apparently the subsequent call within JOGL also succeeds then... I consider that a temporary workaround, not a fix.

There is probably a problem with the class loader. Class.forName() uses the current one, the class loader of the caller. This is not up to JOGL to be in the correct class loader, it is up to the application using it. Just use Thread.currentThread().setContextClassLoader(ClassLoader) if you need to use another class loader like I already do in Eclipse RCP and Netbeans RCP applications. What alternatives do we have? What can we do instead of calling Class.forName()?

I think the Class.forName() is used because OGLUtilities is a private package that you're not normally allowed to import :) These problems do sound like the classloader inside your app is somehow set up strangely (for JOGL, at least).

Do you have any third-party example of a Felix program that uses an external JAR like this? There could be something about your app lifecycle or setup that I'm just unaware of, and it would be nice to know if there's some little Felix-specific trick we're missing.

I'm actually not sure, if Felix is to blame, because if you remember (or take a look at) my first stack trace, I actually wasn't using Felix as OSGi framework at that time, but Eclipse.
I also tested that again. With the small difference, that the Eclipse-OSGi doesn't support the "bundle:/"-protocol-notation, the same error occurs.

For now, my priorities have been shifted, because I needed JOGL just to display a video. Due to the difficulties I'm now implementing a version without OpenGL first.
But that doesn't mean, that I'm not interested in a solution. Unfortunately I haven't had the need myself to customize the Classloader, so I have to read into that first.

sneakpaw wrote

I'm actually not sure, if Felix is to blame, because if you remember (or take a look at) my first stack trace, I actually wasn't using Felix as OSGi framework at that time, but Eclipse.

I know for sure that it will work in Eclipse, because that's how I'm running it  My name.wadewalker.OneTriangle project that I referred to previously is an example of this, and I've tested it on Windows, Mac, and Linux.

I'm sorry, I think I didn't make myself clear.

I'm using bndtools, and within the bnd.bnd (or a bnd-run configuration) I can define which OSGI framework to use: Felix or Eclipse.

All I'm saying is, that I don't think that the problem is in either of them, because with the bnd way of starting them, it doesn't work in either. So probably the issue is connected with bnd, not with Felix.