Java3d pushing the limits...

Hi team,
I haven't contributed much to these forums over the years but I've got a lot out of them and the java3d continuation.

I thought I might share what I've been working on.





https://www.youtube.com/watch?v=MIdjuFqE-dE

It is a project I’ve worked on in my spare time for about ten years now. It is approaching a level where I think people might be interested in seeing what I’ve done, specifically what can be done in pure java, and possibly the code might help someone.

It is a pure java, open source project.

It uses Jogamp’s Java3d for the scene graph and Jogamp’s Jogl for the render bindings https://jogamp.org/

It uses JBullet for physic simulation http://jbullet.advel.cz/

I’ve relied heavily on the NifTools team for decryption of models. http://www.niftools.org/

I am enormously grateful to the above teams/people for what they have done and the entire modding community generally are just awesome.

The binaries can be found here:
https://sourceforge.net/projects/elderscrollsexplorer/
(you need to have access to the game data files)

And finally the source code can be found here:
https://github.com/philjord/ElderScrollsExplorer
 

Hi

Good job!! :D

I didn't know niftools...

Thank you for sharing your source code. Choosing Java3D for a long term project is a bit debatable, I hope that you have another plan. I think that you should use a build tool or a build system to ease the build of your project, it's really complicated... Please choose a license (no license = copyright). I have worked 9 years on my own project and only a few developers use it despite my efforts to make things easier. Good luck.

Julien,
Thanks for checking it out.

Setting licensing was something I wanted to have done before putting it up, but I have an enormous amount of work to sort out the attributions and license statements on the various code I've borrowed and used.

Currently I plan on keeping with Java3d for 2 reasons:

Firstly the scene graph of java3d is still miles ahead of the other options, even though it's 9(+) years since it was designed and written, I am consistently amazing at how forward thinking they were.

Secondly the only major stumbling blocks (once I got compressed textures working) are:

1 My Oculus Rift support (which worked well with SDK 0.3) can't work with the newer SDK and needs JoglPipeline changes (hint hint).
2 I can't get bump mapping going using COMBINE_DOT3 so I have to learn more GLSL (which does my head in frankly).
3 The damn bug in JVM 1.7/1.8 which returns bum drivers on a second init of the canvas, meaning resolution changes require a restart.

Finally, yes I agree furiously about the build system as I wrote that enourmous readme all I could think was it would take the same time to just convert to something, but I know nothing about Maven and the c++ system "make" is such an awful beast I  haven't been able to bring myself to look into it properly. But yes I really should put that on the top of the todo list.

Can I just take a moment to thank you personally for the Jogl port you did, it was an absolute lifeline to me during a time when I nearly gave up on my hobby game coding feeling that it had all been a waste of years of effort. Thank you heaps!

I actually read some of your early post on JGO the other day, amazing to see where you started out and what you've achieved since.

I hope the "discussion" that's just started results in a more lively java3d release path.

I would almost put my hand up for it except:
I'm a father of 3, so I'm unreliable with my time.
I know very little of linux and maven.
I know even less about openGL.
I've got about 6 bug fix/bug examples for Java3d that I still haven't had time to get SSCCE for.

PS I live in Paris, Vive la France!

philjord wrote

Julien,
Thanks for checking it out.

Setting licensing was something I wanted to have done before putting it up, but I have an enormous amount of work to sort out the attributions and license statements on the various code I've borrowed and used.

You should have done that earlier because there are some incompatibilities between licenses...

philjord wrote

Currently I plan on keeping with Java3d for 2 reasons:

Firstly the scene graph of java3d is still miles ahead of the other options, even though it's 9(+) years since it was designed and written, I am consistently amazing at how forward thinking they were.

Java3D has some very deep and impacting design flaws, it's maintained but there will be no new feature, it will never work under Android, it has no chance to work with OpenGL ES and it won't work with Vulkan. We have just postponed its death by maintaining it a bit longer but there are fewer developers contributing on it as time goes by. We don't have enough resource to implement a new backend based on Vulkan without breaking the public API. Java3D isn't miles ahead the JavaFX 3D graphics API, it's a lot less modern than JMonkeyEngine and most of its API goes less far than JogAmp's Ardor3D Continuation which is really designed to be maintained for the next ten years and with JOGL in mind.

philjord wrote

Secondly the only major stumbling blocks (once I got compressed textures working) are:

1 My Oculus Rift support (which worked well with SDK 0.3) can't work with the newer SDK and needs JoglPipeline changes (hint hint).

Java3D is only in maintenance mode, we won't implement any new feature...

philjord wrote

2 I can't get bump mapping going using COMBINE_DOT3 so I have to learn more GLSL (which does my head in frankly).

We all have to learn shader programming one day :)

philjord wrote

3 The damn bug in JVM 1.7/1.8 which returns bum drivers on a second init of the canvas, meaning resolution changes require a restart.

This limitation is absent of JogAmp's Ardor3D Continuation, I just have to move some code from TUER into the engine itself to make it work as flawlessly as possible for the other developers.

philjord wrote

Finally, yes I agree furiously about the build system as I wrote that enourmous readme all I could think was it would take the same time to just convert to something, but I know nothing about Maven and the c++ system "make" is such an awful beast I  haven't been able to bring myself to look into it properly. But yes I really should put that on the top of the todo list.

A portable equivalent of "make" is Ant ;) I saw a few DLLs, it discouraged me to run your stuff under GNU Linux.

philjord wrote

Can I just take a moment to thank you personally for the Jogl port you did, it was an absolute lifeline to me during a time when I nearly gave up on my hobby game coding feeling that it had all been a waste of years of effort. Thank you heaps!

I did most of the port in a few hours (about 4 :D ) and Harvey read all my commits before using them. I don't want you to give up but you'll really have to drop Java3D one day.

philjord wrote

I actually read some of your early post on JGO the other day, amazing to see where you started out and what you've achieved since.

I hope the "discussion" that's just started results in a more lively java3d release path.

I would almost put my hand up for it except:
I'm a father of 3, so I'm unreliable with my time.
I know very little of linux and maven.
I know even less about openGL.
I've got about 6 bug fix/bug examples for Java3d that I still haven't had time to get SSCCE for.

There won't be anything really lively in Java3D. I'll try to make it work with OS 10.11 but I'm reluctant to release Java3D 1.7. You can just expect a few maintenance pre-builds this year and next year. It was designed with no strong reliance on JOGL, this aspect is a weakness, it reinvents the wheel in worse, it's less reliable than the other 3D engines written later.

philjord wrote

PS I live in Paris, Vive la France!

I live extremely close to Paris. I was in the Charonne street one hour and an half before the shooting. I just try to forget what happened.

Your video is really amazing :D

Julien,

Thanks for that feedback.

I just spent a few minutes and discovered team project sets in git, which is exactly what I needed for making the project.

I am familiar with Ant but I don't happen to use it much.

As it happens my entire project only has one build requirement (other than compiling a stack of code) that the tools3d jar file must load before the java3d jar, but it's hard to NOT have that happen.

So now the setup of the code is only a 2 step operation, and a final 2 steps to make it run, so thanks for pushing me to get on with that.

(and it was less time to do that writing and testing the process I had up before in the Readme.md)


I've gone back and checked that at the least all my project have the GLPL in the git repo root, so that's a start on licensing.

I understand completely your thoughts on not enhancing Java3d, once you finally release 1.7 I might put out a little fork that includes compressed texture support, as it's only a few dozen lines of code.


(And all references to dll,c++, visualstudio and that sort of madness are now gone...)

philjord wrote

I understand completely your thoughts on not enhancing Java3d, once you finally release 1.7 I might put out a little fork that includes compressed texture support, as it's only a few dozen lines of code.

We can put it into Java3D 1.6 but someone will have to test your code with some compressed textures to be sure that it works. I would prefer to discourage any fork. There is only one Java3D and it won't change anytime soon. Forking Java3D would only lead to confusion.

Julien,
Your comment on JGO made me want to prove it was possible to crash JOGL just by the order of calls on Java3D.

I have discovered two things:
1. That it is possible
2. The javadocs for the call that does it is ridiculously clear that it will destroy the world.

So I can't call you out on it, I can't even create an issue in bugzilla, I have accept it was my own failure to debug my code.

On the bright side; a bug that's been annoying me for a long time and made my game crash to desktop just by clicking a menu button is fixed now. woot!

In case anyone else is reading this forum post and has a Java3D crash (with some text about an old driver) and needs to know the cause (using jre 1.7+)

I was doing this on the JFrame containing my Canvas3D
 
        frame.removeNotify();
        frame.setUndecorated(true);
        frame.addNotify();

And a moments review of the javadoc, shows how wildly wrong it is to call that code.
Makes this Frame undisplayable by removing its connection to its native screen resource. Making a Frame undisplayable will cause any of its children to be made undisplayable. This method is called by the toolkit internally and should not be called directly by programs.




So thanks for making me debug my own code.

Now I'm gonna make java3d run on android...

Hi

When you destroy the native peer and/or when you remove any AWT/Swing JOGL canvas/panel from its container, the OpenGL context is destroyed.

philjord wrote

Now I'm gonna make java3d run on android...

Java3D tightly relies on AWT which isn't available under Android. You'll have to implement a completely different code path based on NEWT.

Java3D heavily uses the fixed pipeline. It uses some OpenGL calls not available even in OpenGL ES 1. It can't work as is with OpenGL ES 2. Android uses OpenGL ES. You'll have to use GLSL in a brand new rendering code path to work with OpenGL ES 2.

You wrote that you are busy with your three children and that you don't really know OpenGL (and OpenGL ES?) and GLSL but you claim that you're going to make Java3D run under Android. I don't understand your position. If porting Java3D to Android was trivial, we would already have done it. Canvas3D cannot be used as is under Android. Do you see what I mean? I see a huge contradiction: if you don't want to learn OpenGL, OpenGL ES and GLSL, you'll have no chance of porting any decent scenegraph API for Android.

My final statement was an example of self-deprecating irony.

As a native French speaker I'm not surprised you took it literally, Europeans tend to speak their minds fairly directly. The English however(and native English speakers) tend to say things in a far more circumspect manner.

This is why directly telling someone they are wrong (in English) is the literary equivalent of saying "Fu*k you", and that is generally the way it will be taken.

Or to put it in the reverse, when some says (for example) "I think you may have got that confused..." they are really saying "You are way wrong and I'm going to tell you the truth now..."

Just some thoughts...

Ok, your contributions are still very welcome anyway.

I'd just like to say that Jogamp's Newt windowing toolkit is incredible, why do more people not know about this?

Here's an example of what can be done converting from AWT to Newt in about 10 minutes

Great job! Are you going to port your code away from Java3D?

In a manner of speaking, I've forked Java3D and "converted" it across to ES2, Newt and Curve.

I'm getting close to trying in in Android Studio.

I looked at ways to change scene graph, but they all came out as more work for my project than just fixing Java3D (or Java3Des as I should probably call it).

Jogl, Newt and Curve really are truly fantastic tools, though as everyone says the locations of some files can be hard discover.

Forking Java3D whereas it is still actively maintained isn't very smart in my humble opinion but anyway, as it's under GPL, you'll have to publish your fork under the same license and I'll be happy to integrate your changes back into Java3D. There can only be one Java3D and I won't allow anybody to benefit of the fame of this scenegraph to confuse the developers. Please don't use the term java3des for your fork and rather contribute to Java3D.

Julien,
You make fair points.

Forked should have been in quote marks, I haven't done a real fork, just taken the code in my private repo and hacked it up
until it compiled without rt.jar and with the GL2ES2 profile.

Rest assured I am definitely not going to supply yet another scene graph or a fork of a scene graph or a plugin in or whatever,the java gaming world has plenty enough as it stands now.

I'm also not going to use the Java3D name in any manner except where I sing it's praises.
I love Java3D and I want other people to see how good it is at it's job.

Right now my code is a shambles, so there would be literally no point in donating it back to Java3D as it stands, it would only waste your time to even look at it.
I suspect a few more months of clean up might get it into a readable state.

Having said the above I feel the Pipeline part of what I've done to support non-FFP could possibly one day be integrated back into Java3D, which might be useful if GPUs ever start to drop GL2 support. I haven't looked closely but one of my earlier commits might not be too far from being able to be another Pipeline option.

The non rt.jar side of things is wildly incompatible with core Java3D, so I don't think it would/could be integrated, short of some violently major work
to pull the common parts out and then optionally branch to awt or newt. As you say the Java3D API is frozen.

Finally, I have read the license of Java3D and on the day that I need to distribute my code, presumably being some time shortly after someone gives me some money for my hobby (this seems unlikely), on that day the problem of distributing my source will be a nice problem to have.

Thanks for the feedback, once again you force me to think and consider things carefully, which helps me tremendously to understand what I'm doing and why.

Phil.

I don't want to be harsh.

We can work together in order to find a way of integrating your changes without breaking the existing API or we can do those breaking changes in Java3D 1.7.0 (not in Java3D 1.6.0). By default, Java3D will use the fixed pipeline except in cases where it has no chance to work but a pipeline option is a good idea to me.

Why do you mean by "the non rt.jar side"?

The problem is that the new pipeline will require some maintenance on the long term. Your changes will be helpful but it won't eliminate all design flaws of Java3D.

By non rt I'm refering to my attempts to get java3d to run on android.
First off I had to make a non ffp ES2 pipeline, which I'll contribute back.

Then I had to compile java3d with javas rt.jar swapped for android.jar.

As you can imagine this broke about half of java3d especially the canvas3d and all image classes.

So I just kept hacking away until it compiled, by using newt, curve and all sorts of crazy stuff.

So that side of what I've done could only be merged back into java3d with a staggering level of effort.

So I described it as java3dES to suggest it's permanently branched.

Obviously I'm still required to release it so I will at some point, but I think it would only cause terrible confusion if it gets associated with java3d in any manner, so when I publish it I'll do it very quietly in my github account, and indicate to people it's almost certainly not what they want.

Hope that clarifies thibgs a bit,
Phil

Thank you for this clarification. It's totally possible to make a separate sub-project for Android specific stuff, there is already java3d-util, there was ardor3d-android several years ago. In my humble opinion, the NEWT stuff not depending on Android should be put into Java3D itself and not into a separate sub-project. Java3DES isn't a good name for something supporting Android, java3d-android is more consistent to me.

I'm especially interested in the pipeline changes you've made as all of the infrastructure for multiple pipelines still exists in Java3D, and if you're willing to contribute your changes upstream I'd be very interested in trying to integrate them.

Harvey