appbundler jogl jnilib dylib one-jar confusion

Hi,

Now this might be one-jar or Mac Java 7 problem so I will be
pestering those lists too but this is so confusing that
I want to left no stone interned...

Ok, to the business, here it the weirdest thing that
has been bothering me on and off for a few weeks.

I'm developing an application that uses OpenGL via
JOGL. I package it using one-jar and then use launch4j
and appbundler to turn it into a .exe and .app file/bundle.

I'm pretty sure this was working flawlessly some months ago,
I might believe I had dreamed but there is this dialog
from my in the web so I must have had it working previously:

http://stackoverflow.com/questions/8943804/launch4j-onejar-packed-exe-file-corrupted

Anyway, now all of a sudden when I came back to this
project after summer, the .app refused to start, it launched
and but nothing happened. Starting from Terminal I saw
that it had trouble loading jnilib's for JOGL.

In the interim time what has changed is that I've upgraded
to Java 7 and Mountain Lion (and Retina MacBook Pro).

Figuring this might be that JOGL/jogamp or one-jar had gone stale
I've upgraded to the latest.

Now here were it gets confusing.

In Eclipse, to be able to launch the app it is enough to include:

gluegen-rt-natives-macosx-universal.jar
gluegen-rt.jar
gluegen.jar
jogl.all.jar

in the project build path. Works like a charm.

To be able to use one-jar I need to include not only:

gluegen-rt.jar
gluegen.jar
jogl.all.jar

but the native libraries unpacked from the jar:

libgluegen.rt.dylib
libjogl_desktop.dylib
libnativewindow_awt.dylib
libnativewindow_macosx.dylib

and interestingly/confusingly these need to be .dylib
not .jnilip although the jogamp distrubutes them as
.jnilib and apparently Eclipse can figure this out
when those files are in the gluegen-rt-natives-macosx-universal.jar

What's going on?

Anyway with above I can get one-jar to produce a single
jar that works fine, apart from having the correct process
name for the application and a nice icon.

So I apply appbundler to that jar and turn it into a .app bundle
and again I'm back where I started:

nyholku$ EazyCNC.app/Contents/MacOS/JavaApplicationStub
Loading configuration from /Users/nyholku/EazyCNC/EazyCNC-Mach-Config.ecnc
Catched UnknownHostException: Users, while TempJarCache.bootstrapNativeLib() of jar:file://Users/nyholku/EazyCNCProject/bin/EazyCNC.app/Contents/Resources/Java/EazyCNC.jar!/lib/gluegen-natives-macosx-universal.jar!/ (file://Users/nyholku/EazyCNCProject/bin/EazyCNC.app/Contents/Resources/Java/EazyCNC.jar!/lib/ + gluegen-natives-macosx-universal.jar)
java.lang.reflect.InvocationTargetException
        at java.awt.EventQueue.invokeAndWait(EventQueue.java:1111)
retrigCompile eazycnc.GCodeEditorPanel$5.run(GCodeEditorPanel.java:173)
......
        at apple.launcher.LaunchRunner.run(LaunchRunner.java:116)
        at apple.launcher.LaunchRunner.callMain(LaunchRunner.java:51)
        at apple.launcher.JavaApplicationLauncher.launch(JavaApplicationLauncher.java:52)
Caused by: java.lang.UnsatisfiedLinkError: no gluegen-rt in java.library.path
        at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1758)
        at java.lang.Runtime.loadLibrary0(Runtime.java:823)
        at java.lang.System.loadLibrary(System.java:1045)
...

So several weird things going on here, any ideas to go forward would
be appreciated.

To recap:

Why Eclipse works with just the .jar files and/or why
the one-jar classloader does not?

Should the native libraries be .jnilib or .dylib? I've
always used .jnilib when I've called native code from Java?

Why the appbundler packaged working .jar fails to load
the native libraries?



br Kusti

Hi

There is no confusion. gluegen.jar is never required whereas gluegen-rt.jar is necessary. The automatic extraction and loading of native libraries from JARs seem to work in Eclipse but not in One-JAR because the latter does not know the native libraries are in some JARs.

Maybe consider using Java Web Start, IzPack, GetDown or any other installer.

On 24.11.2012 13.26, "gouessej [via jogamp]"
<[hidden email]> wrote:

Hi Julien,

thanks for answering, I appreciate it.

Your answer clarifies some of the issues but still some remain:

Should it be .dylib or .jnilib with JOGL on Mac?

For this particular application Web Start is not an option
though I like it a lot and use it in an other application.

Thanks for pointing out IzPack and Getdown. I may end up
using one of those for Linux and Windows but Mac users
shun installers and don't want to hear about them. What
they want is an .app bundle and that is what is not
working for me. I need to figure out why appbundler
bundled single .jar file that works by itself does not
work when it is launched with appbundlers  JavaApplicationStub.

best regards and thanks, Kusti

>

The use of .dylib and .jnilib depends on the version of Mac but I'm not comfortable with Macs, maybe Sven can confirm.

I'm even not sure bundling an application based on JOGL as a fat JAR is still working on other platforms, someone else provided a fix but we didn't understand it and we preferred not using it.

On 11/24/2012 02:18 PM, gouessej [via jogamp] wrote:
> The use of .dylib and .jnilib depends on the version of Mac but I'm not
> comfortable with Macs, maybe Sven can confirm.
>

We [still] use .jnilib as the suffix and it works on Apple's provided old JRE
and on the new OpenJDK for OSX.

Yes, the new OpenJDK, or Java7, JRE is able to also load OSX's native .dylib suffix.

> I'm even not sure bundling an application based on JOGL as a fat JAR is still
> working on other platforms, someone else provided a fix but we didn't
> understand it and we preferred not using it.

Indeed.

We have to be very careful w/ GlueGen patches, since it has potential
to break it for all modules and platforms.

For example, if you provide an OSX patch for GlueGen regarding native lib loading etc -
you shall test on at least the old and new JREs.

Ofc .. we also need to discuss it and some sort of documentation/explanation
is essential to grok the code.

This has been criticized as arrogant [at least] already, however,
I hope you all understand the impact such patches may have .. etc etc :)

~Sven

> Julien Gouesse
> http://tuer.sourceforge.net
> http://gouessej.wordpress.com
>

A note on the .dylib vs. .jnilib problem -- I ran into this problem earlier this year with JDK 7 on Mac OS X. The root of the problem is that Oracle changed System.mapLibraryName() in JDK 7 to look for libraries using the .dylib extension instead of the .jnilib extension (see http://mail.openjdk.java.net/pipermail/core-libs-dev/2012-April/009725.html and https://bugs.eclipse.org/bugs/show_bug.cgi?format=multiple&id=276564 for details).

This means that in Eclipse plugins (which find native libraries using class loader hooks that eventually call System.mapLibraryName()), it will look for <library name>.dylib. But in JOGL's native library loader, we explicitly load the library using System.loadLibrary( <library name>.jnilib ), so .jnilib still works in that case even with JDK 7.

My tutorial at http://wadeawalker.wordpress.com/2010/10/09/tutorial-a-cross-platform-workbench-program-using-java-opengl-and-eclipse/ makes note of this, since it shows how to include the native libraries in an Eclipse plugin.

On 24.11.2012 15.18, "gouessej [via jogamp]"
<[hidden email]> wrote:

>
>
>    The use of .dylib and .jnilib depends on the version of Mac but I'm
>not comfortable with Macs, maybe Sven can confirm.
>
>
>I'm even not sure bundling an application based on JOGL as a fat JAR is
>still working on other platforms, someone else provided a fix but we
>didn't understand it and we preferred not using it.


Well, the fat jar approach on Mac OS X is working at least, but
the .jnilib files need to be renamed to .dylib at least for Java 7.
Have not tested Java 6 as that is dying fast and the future of
Java on Mac OS X is embedding the JRE with your app so no need
to support older version on Mac (what I mean that since I will
be delivering the as a 'proper' mac application bundle I don't
need to support old JRE version.

Some more info.

First I need to correct false information I gave. I was not
using 'appbundler' but 'jarbundler'. When I realized that
I switched over to 'appbundler' and now I am able create
a fully working Mac OS X bundler with or without embedded
JRE and with JOGL and my app and all libraries in a single
fat jar which I created with one-jar.

br Kusti

On 24.11.2012 22.05, "Wade Walker [via jogamp]"
<[hidden email]> wrote:


>This means that in Eclipse plugins (which find native libraries using
>class loader hooks that eventually call System.mapLibraryName()), it will
>look for <library name>.dylib. But in JOGL's native library loader, we
>explicitly load the library using System.loadLibrary( <library
>name>.jnilib ), so .jnilib still works in that case even with JDK 7.


I'm sure you are right, but for me this appears to work
exactly opposite.

In Eclipse I use just the jar files (which contains
the .jnilib files) and that works.

But when I'm using fat jar I create with one-jar
I need to use the .dylib renamed natives.

So yes you are right, but for the casual observer
this might look just the opposite. Confusing.

And many thanks for the answer and pointer to the
Oracle change. Interesting times we are living
in Java world, so many things moving all at once.

br Kusti

Hi,

like I reported the fat jar approach works for me on Windows and Mac OS X,
but on Ubuntu I get:

Catched UnknownHostException: home,
while TempJarCache.bootstrapNativeLib() of jar:
file://home/nyholku/Desktop/./EazyCNC!/lib/gluegen-natives-linux-i586.jar!/
 
(file://home/nyholku/Desktop/./EazyCNC!/lib/ +
gluegen-natives-linux-i586.jar)
Catched UnknownHostException:
home, while addNativeJarLibsImpl(classFromJavaJar class
jogamp.nativewindow.NWJNILibLoader, classJarURL
jar:file://home/nyholku/Desktop/./EazyCNC!/lib/jogl-all.jar!/jogamp/nativew
indow/NWJNILibLoader.class, nativeJarBaseName jogl-all):
jogl-all-natives-linux-i586.jar +
file://home/nyholku/Desktop/./EazyCNC!/lib/ ->
jar:file://home/nyholku/Desktop/./EazyCNC!/lib/jogl-all-natives-linux-i586.
jar!/
Catched IOException: TempJarCache: addNativeLibs:
jar:file://home/nyholku/Desktop/./EazyCNC!/lib/jogl-all-natives-linux-i586.
jar!/, previous load attempt failed, while
addNativeJarLibsImpl(classFromJavaJar class
jogamp.nativewindow.NWJNILibLoader, classJarURL
jar:file://home/nyholku/Desktop/./EazyCNC!/lib/jogl-all.jar!/jogamp/nativew
indow/NWJNILibLoader.class, nativeJarBaseName jogl-all):
jogl-all-natives-linux-i586.jar +
file://home/nyholku/Desktop/./EazyCNC!/lib/ ->
jar:file://home/nyholku/Desktop/./EazyCNC!/lib/jogl-all-natives-linux-i586.
jar!/
Catched IOException: TempJarCache: addNativeLibs:
jar:file://home/nyholku/Desktop/./EazyCNC!/lib/jogl-all-natives-linux-i586.
jar!/, previous load attempt failed, while
addNativeJarLibsImpl(classFromJavaJar class
jogamp.nativewindow.NWJNILibLoader, classJarURL
jar:file://home/nyholku/Desktop/./EazyCNC!/lib/jogl-all.jar!/jogamp/nativew
indow/NWJNILibLoader.class, nativeJarBaseName jogl-all):
jogl-all-natives-linux-i586.jar +
file://home/nyholku/Desktop/./EazyCNC!/lib/ ->
jar:file://home/nyholku/Desktop/./EazyCNC!/lib/jogl-all-natives-linux-i586.
jar!/


So I'm left wondering where to start looking for trouble (oops, looks
like I'm already looking for trouble with a fat jar approach ;-)?

If I understood what the exception says some one (is that JOGL stuff?)
is trying to find host 'home' ... AFAIU there URL is wrong as it
if the host is omitted then the there should be /// not // and
thus it is technically correct that host 'home' cannot be found....?!

Also I'm wondering what is that exclamation number in 'EazyCNC!'
and in 'gluegen-natives-linux-i586.jar!' ?

Who/what constructs that URL?

Is this piece just some helpfull output

(file://home/nyholku/Desktop/./EazyCNC!/lib/ +
gluegen-natives-linux-i586.jar)


as it looks even less valid if interpreted as an URL?


best regards Kusti

The fat jar approach is a really bad idea under Windows and GNU Linux because the archiver can open it instead of Java, lots of people use WinRAR under Windows and on other Linux distros some people use Ark.

I think that you have the same problem under Ubuntu and Mac OS X. Maybe we can fix this bug but I still don't recommend this way of deploying applications based on JOGL 2.0.

On 26.11.2012 17.39, "gouessej [via jogamp]"
<[hidden email]> wrote:

>
>
>    The fat jar approach is a really bad idea under Windows and GNU Linux
>because the archiver can open it instead of Java, lots of people use
>WinRAR under Windows and on other Linux distros some people use Ark.

Ack that, but I've turned the fat jar to .exe with launch4j and it works
nicely
and then we should not have this problem you mention.

>
>
>I think that you have the same problem under Ubuntu and Mac OS X. Maybe
>we can fix this bug but I still don't recommend this way of deploying
>applications based on JOGL 2.0.


Do you meant that I need to unpack the .so files?

I tried that and it did not make a difference?

Still wondering about badly formatted URL ... looks like if
the URL had a correct format this might have actually worked?!

Thanks for responding.

br Kusti

nyholku wrote

On 26.11.2012 17.39, "gouessej [via jogamp]"
<[hidden email]> wrote:

>
>
>    The fat jar approach is a really bad idea under Windows and GNU Linux
>because the archiver can open it instead of Java, lots of people use
>WinRAR under Windows and on other Linux distros some people use Ark.

Ack that, but I've turned the fat jar to .exe with launch4j and it works
nicely
and then we should not have this problem you mention.

Yes you should not have the problem I mentioned but if something goes wrong with launch4j, we won't be able to help you, especially if you use a non standard embedded JVM. It would be less risky to use Java Web Start.

nyholku wrote

>
>
>I think that you have the same problem under Ubuntu and Mac OS X. Maybe
>we can fix this bug but I still don't recommend this way of deploying
>applications based on JOGL 2.0.


Do you meant that I need to unpack the .so files?

I tried that and it did not make a difference?

Still wondering about badly formatted URL ... looks like if
the URL had a correct format this might have actually worked?!

Thanks for responding.

br Kusti

It's worse than I thought. I thought unpacking the .so files would work.

On 26.11.2012 18.25, "gouessej [via jogamp]"
<[hidden email]> wrote:
>
>
>Yes you should not have the problem you mention but if something goes
>wrong with launch4j, we won't be able to help you, especially if you use
>a non standard embedded JVM. It would be less risky to use Java Web Start.


Yes, I understand that you might not be able to help me.

Just to clarify:

- Web Start is not acceptable for this application I'm afraid.

- I'm not planning to use a non standard JRE.

- I may have to embed a standard JRE to guarantee compatibility
and behaviour. This seems to imply that it is not possible to
use a fat jar approach by itself as the JRE cannot reside in
the fat jar right?

So I may end up using an installer of some kind approach,
do you have any recommendations in that respect?


>
>
>
>It's worse than I thought. I thought unpacking the .so files would work.

So far no, but I'll keep tinkering around.

br Kusti

On 11/27/2012 12:44 PM, nyholku [via jogamp] wrote: Hi, sorry - I have missed the whole discussion.

Ok, so we have the following deployment models working:
  - JNLP webstart (Applets, Application, ..)  

  - NApplet (Applet's using our build-in native lib loading)

  - Applications using our build-in native lib loading  

  - Jar-In-Jar
    - <https://jogamp.org/bugzilla/show_bug.cgi?id=522#c7>
    - <http://jogamp.org/git/?p=gluegen.git;a=commit;h=04391a3f417e10e1b6dafbd8becc63659af633c3>

All the above have the following in common,
the native library of one module (gluegen, jogl) shall reside in a jar file
at the same base URI of the module with the derived 'natives-OS.and.ARCH' name.

Example:
   file:///lala/gluegen-rt.jar
   file:///lala/gluegen-rt-natives-linux-amd64.jar
   file:///lala/jogl-all.jar
   file:///lala/jogl-all-natives-linux-amd64.jar

It doesn't matter whether the JAR basename is kept original or is modified,
we have fixed this w/ our maven2 support.

For example, you can also repack all-in-two (gluegen, jogl, your-app):
   file:///lala/MyApp.jar
   file:///lala/MyApp-natives-linux-amd64.jar

Please note that above examples only name linux-amd64,
but you should add all native JAR files to support all platforms.

IMHO it would suit you best to use above Jar-In-Jar method,
since it will keep the original JogAmp files intact incl. version numbers.
The latter will help producing proper bug reports etc - but whatever floats your boat.

While we _DO_IGNORE_ all products doing some magic like bundling etc,
our described deployment mechanism is as flexible as it can get
and should be easily applied with your model!

~Sven

Brilliant, thanks, I'll have a look at those and report back
if something works for me (or not).

br Kusti



On 27.11.2012 14.02, "Sven Gothel [via jogamp]"
<[hidden email]> wrote:


--
Kustaa Nyholm
Research Manager, Software
Research and Technology Division
PLANMECA OY
Asentajankatu 6
00880 HELSINKI
FINLAND

Please note our new telephone and fax numbers!
Tel: +358 20 7795 572 (direct)
Fax: +358 20 7795 676
GSM: +358 40 580 5193
e-mail: [hidden email]