Java 3D: Use Maven to build, and publish Maven artifacts

Hi everyone,

At Sven's request, I am posting here about the recent work I did to Mavenize the Java 3D 1.6 codebase.

Mavenizing the codebase

The relevant issue on GitHub is hharrison/java3d-core#14: an issue about publishing Maven artifacts to Maven Central, but there is a related-but-separate issue of whether to Mavenize the Java 3D build system.

I personally believe we should do so, so I did it:

• ctrueden/java3d-core@maven
• ctrueden/java3d-utils@maven
• ctrueden/vecmath@maven

In order to complete the Mavenization, it was necessary to break the circular dependency between java3d-core and java3d-utils (see hharrison/java3d-core#17). PRs were filed for that work:

• hharrison/java3d-core#19
• hharrison/java3d-utils#2

And I filed a PR for the Mavenization of vecmath:

• hharrison/vecmath#5

These patches are all still awaiting review from Harvey.

The SciJava fork

Because it has been 8 months since I did the Mavenization of the Java 3D components, and there did not seem to be forward progress upstream, I (temporarily!) forked the components to my project's SciJava organization on GitHub:

• scijava/java3d-core
• scijava/java3d-utils
• scijava/vecmath

I also published unofficial builds to Maven Central. I marked these clearly as unofficial according to Maven best practice, by changing the groupId and adding a version suffix.

I want to be clear that it was not my intent to create a permanent fork of Java 3D—I would much rather contribute changes upstream. But the JOGL version of Java 3D has been around for years now and I thought the community would benefit from a way to depend on it using industry standard tools, even unofficially for the time being.

To those responsible for maintaining these components: I greatly appreciate all your time and effort. Just let me know what I can do to help move things forward.

Regards,
Curtis

On 10/22/2015 09:05 PM, ctrueden [via jogamp] wrote:
> Hi everyone,
>
> At Sven's request, I am posting here about the recent work I did to Mavenize
> the Java 3D 1.6 codebase.
>

Thank you Curtis.

I guess we can give Harvey a few more days to reply,
then we shall go forward somewhat.

@Harvey: Shall we ask for one additional maintainer for Java3D?
@Julien: I guess this procedure will suit all of us.

[1] The maintainer shall merge the work if fit
and push them to our official git repository
at: http://jogamp.org/git/

[2] Then the maintainer shall produce builds
and drop them to: http://jogamp.org/deployment/java3d/
(inclusive the maven artifacts !?)

[3] Last but not least, Mark shall be notified
to push those artifacts to maven central.

As it is true for all maintainer, they shall be fluent in:
  - git basic operations
  - security and ssh
  - know the beast (i.e. the module they are taking care of)

the maintainer needs to have [limited] access to
  - jogamp.org via ssh
  - communicate

So lets wait for Harvey and Julien
and see who likes to take care of these tasks.

Thank you Curtis for your work!

Cheers, Sven

Proposed maven namespace is
  org.jogamp.java3d.*

i.e. for all three modules.

Sven Gothel wrote

Proposed maven namespace is org.jogamp.java3d.* i.e. for all three modules.

Makes total sense. I updated the maven branch of each component to use the org.jogamp.java3d groupId.

There are other updates that will likely need to happen to the Maven POMs for everything to be fully kosher, which I discovered during my efforts to push to OSS Sonatype yesterday:

• Add project name, description, URL and SCM sections.
• Update the developers block to be more accurate, and maybe list developer roles.
• Change ${project.version} dependency versions to explicit release versions -- if we care about reproducible SNAPSHOT builds. Pros and cons here though.

I can make an effort but I'd like Mark to look carefully at the Maven stuff too as he's more experienced than me with this build system.

First off, my apologies for the lack of time/energy to steward this project. I think at this point if I can't find the time to properly evaluate your changes, I may just pull them as-is to get some movement here, while I'm not totally comfortable with Maven, I think I can live with it as long as someone else is there to review build changes.

So, poke me in a week or so if nothing has changed in my repos, and I will merge your request as is at that point.

Fair?

Harvey

Thanks for the response, Harvey. :-)

hharrison wrote

while I'm not totally comfortable with Maven, I think I can live with it as long as someone else is there to review build changes.

No worries!

I think the main thing to verify is that the compiled binaries match the project's requirements, when compared to the Ant builds. Are all the property files and values present and correct, META-INF info, version constants in the source, etc. I did do some of that checking (e.g. "jar tf j3dcore-ant.jar | sort" diffed against "jar tf j3dcore-maven.jar | sort"), but it would be nice for someone more familiar with Java 3D to verify my work.

I'm certainly happy to follow up with any further build system fixes, improvements, advice, etc. at a later date.

hharrison wrote

So, poke me in a week or so if nothing has changed in my repos, and I will merge your request as is at that point.

Fair?

Sounds good!

@hharrison wrote:
> poke me in a week or so

Ping.

Also, there is a new wrinkle, which I am struggling with how to solve, and I would value input from the JogAmp community.

OS X 10.11 "El Capitan" has a new feature, System Integrity Protection (SIP), a.k.a. "rootless" mode, which disallows modification to various folders, including /System, even as root.

Unfortunately, /System/Library/Java/Extensions houses the obsolete Java 3D 1.3 JARs + associated native libs, which when invoked from Oracle Java 8, crash the JVM. And even with Java 3D 1.6 libraries on the classpath, the old Java 3D 1.3 always takes precedence because /System/Library/Java/Extensions is on OS X's java.ext.path by default.

I thought of two different ways to address this new difficulty:

1. Shade the package prefix(es) of Java 3D 1.6 to avoid a clash.
2. Manually manipulate the java.ext.path.

Option 2 is a deployment-centric solution, which only solves the problem in environments where tweaking the Java runtime parameters is possible. It is also a pain because there is no way to simply blacklist the undesirable directory; instead, the java.ext.path must be built from the ground up to include all the directories which Java includes by default, except for /System/Java/Library/Extensions. Furthermore, there _are_ libraries in said directory that might continue to be useful for ImageJ users, such as the support for AppleScript. It would be a shame to lose access to _all_ Apple-specific Java extensions, just because of three bad apples (the old Java 3D 1.3 JARs).

It seems to me that choosing a new package prefix would make more sense. It would mean that legacy code needs to be refactored, but the changes would be trivial updates to import statements, easily accomplished via global search-and-replace.

Does anyone see anything I'm missing here? Any other possible ways to avoid the Java 3D OS X apocalypse?

+1 for changing the package prefixes. Yes it breaks compatibility with old source code but I think it is worth it.

Actually I don't use java3D myself but I still use vecmath in all my 3D projects. An 'official' jogamp vecmath jar in maven central would be pretty sweet :-)

Hi

I'm not in favour of a change in the package naming because this is typically what we have tried to avoid for years in order to stay compatible with earlier versions of Java3D. It's possible to modify the classpath at runtime, it's not a very clean solution but changing everything only because a problem on one operating system is nonsensical to me.

I have thought about your problem...

I suggest the following solution, I'd like to get Harvey's opinion:
- Java3D 1.6 pre 13 remains frozen with a few bugs and this limitation under OS X
- Java3D 1.7 uses another namespace instead of "javax.media" and includes some goodies (more type safe methods, relies on Java 1.7, drops support of Java 1.6)

Then, the developers who would like to support OS X 10.11 would be forced to switch to Java3D 1.7.

However, ctrueden, note that the extension mechanism has been dropped in Java 1.9, you'll have to make some other changes to go on using AppleScript. That's why another solution consists in changing nothing within Java3D. You can bundle your own JRE with your software.

I'll have to update my tutorial very soon.

Hi Julien,

> the extension mechanism has been dropped in Java 1.9, you'll have to
> make some other changes to go on using AppleScript.

AppleScript was just an example—the main point is that Oracle Java adds /System/Library/Java/Extensions to the java.ext.path by default, which makes it impossible to use Java 3D 1.6. And manually controlling the java.ext.path:

1. Might not be possible in some deployment environments; and
2. Requires you to manually specify all other needed directories—in particular, the JRE's lib/ext folder.

> That's why another solution consists in changing nothing within
> Java3D. You can bundle your own JRE with your software.

Bundling your own JRE is insufficient on OS X, because your bundled Oracle JRE adds /System/Library/Java/Extensions to the extensions path by default, preventing use of Java 3D 1.6.

> - Java3D 1.6 pre 13 remains frozen
...
> Java3D 1.7 uses another namespace instead of "javax.media"

I really like this idea! However, given the rate things are going with Java 3D these days, I cannot afford to hold my breath for Java 3D 1.7—nor even to invest the energy required to push for it over the next months.

I receive multiple bug reports each month relating to Java 3D + Java 8 incompatibility; users need it fixed ASAP. So for the time being, I will keep maintaining the org.scijava fork, with updated package prefixes: org.scijava.java3d and org.scijava.vecmath.

In the meantime, if there anything I can do to help move Java 3D forward on the Maven front etc., the JogAmp team is welcome to contact me via email.

On 11/18/2015 04:52 PM, ctrueden [via jogamp] wrote:
> I really like this idea! However, given the rate things are going with Java 3D
> these days,

I agree here .. and if Harvey also agrees,
maybe somebody else can take the lead role for our Java3D?
This is not a demotion, just a practical approach of the current situation,
as all our 'roles' are just a condensation of reality :)

Sure .. we would need somebody taking that role.
Volunteers are welcome!

> I cannot afford to hold my breath for Java 3D 1.7—nor even to
> invest the energy required to push for it over the next months.
Sure.

>
> I receive multiple bug reports each month relating to Java 3D + Java 8
> incompatibility; users need it fixed ASAP. So for the time being, I will keep
> maintaining the org.scijava fork, with updated package prefixes:
> org.scijava.java3d and org.scijava.vecmath.

if it works for you ..

>
> In the meantime, if there anything I can do to help move Java 3D forward on
> the Maven front etc., the JogAmp team is welcome to contact me via email.

I myself don't want to take the above role,
but as I mentioned .. you yourself could also take that role,
which you rejected last time we had a chat :)

So .. volunteers may step forward for Java 3D!
  - merge things
  - communicate
  - push deployments

Cheers, Sven

Hi

ctrueden wrote

Hi Julien,

> the extension mechanism has been dropped in Java 1.9, you'll have to
> make some other changes to go on using AppleScript.

AppleScript was just an example—the main point is that Oracle Java adds /System/Library/Java/Extensions to the java.ext.path by default, which makes it impossible to use Java 3D 1.6. And manually controlling the java.ext.path:

1. Might not be possible in some deployment environments; and
2. Requires you to manually specify all other needed directories—in particular, the JRE's lib/ext folder.

It's possible to remove some JARs from the classpath at runtime but it must be done very early. The point is OpenJDK and Oracle Java 1.9 have no notion of extension mechanism anyway. Another solution consists in stopping to rely on it right now by adding the JARs that you were accustomed to refer to under the hood explicitly into your classpath and setting java.ext.path to "". I can help you to fix a few corner cases. What do you think about this solution?

ctrueden wrote

> That's why another solution consists in changing nothing within
> Java3D. You can bundle your own JRE with your software.

Bundling your own JRE is insufficient on OS X, because your bundled Oracle JRE adds /System/Library/Java/Extensions to the extensions path by default, preventing use of Java 3D 1.6.

You can use OpenJDK 1.9.

ctrueden wrote

> - Java3D 1.6 pre 13 remains frozen
...
> Java3D 1.7 uses another namespace instead of "javax.media"

I really like this idea! However, given the rate things are going with Java 3D these days, I cannot afford to hold my breath for Java 3D 1.7—nor even to invest the energy required to push for it over the next months.

I receive multiple bug reports each month relating to Java 3D + Java 8 incompatibility; users need it fixed ASAP. So for the time being, I will keep maintaining the org.scijava fork, with updated package prefixes: org.scijava.java3d and org.scijava.vecmath.

In the meantime, if there anything I can do to help move Java 3D forward on the Maven front etc., the JogAmp team is welcome to contact me via email.

I can help you to find a satisfying workaround. Blacklisting the old craps would allow to solve your problem within Java3D without any major changes. Where is your bug tracker?

ctrueden wrote

So for the time being, I will keep maintaining the org.scijava fork, with updated package prefixes: org.scijava.java3d and org.scijava.vecmath.

In the meantime, if there anything I can do to help move Java 3D forward on the Maven front etc., the JogAmp team is welcome to contact me via email.

I can provide a first blueprint of Java3D 1.7 Saturday or Sunday but I expect you to abandon your fork in this case. It's just a matter of a few hours for me.

I think that I'll include your fixes directly in Java3D 1.6.0 as soon as possible. I'll do the package name changes in Java3D 1.6.1 pre 1. I really want to get something working very soon so that I can freeze the API and release a stable version.

Hi

Thanks to ctrueden, we can build vecmath with Maven :) Big kudos to him :D You can look at my repository on Github to test the changes:
https://github.com/gouessej/vecmath

gouessej wrote:
> we can build vecmath with Maven :)

Next steps for the Mavenization of Java 3D:

1) Break the circular dependency between java3d-core and java3d-utils. This is done and filed as PRs:

* https://github.com/gouessej/java3d-core/pull/1
* https://github.com/gouessej/java3d-utils/pull/1

Original discussion of the problem can be seen at:

* https://github.com/hharrison/java3d-core/issues/17

2) Mavenize the java3d-core and java3d-utils builds. This is done on respective branches:

* https://github.com/gouessej/java3d-core/compare/pull/1/head...scijava:maven
* https://github.com/gouessej/java3d-utils/compare/pull/1/head...scijava:maven

I'll file PRs for those as soon as the PRs for the circular dependency are merged.

3) Choose a new package prefix to use starting with Java 3D 1.7. Very likely "org.jogamp.java3d"?

4) Migrate the repositories to the jogamp organization and add more metadata to the POMs (url, scm, issueManagement).

Sven Gothel wrote:
> I myself don't want to take the above role,
> but as I mentioned .. you yourself could also take that role,
> which you rejected last time we had a chat :)

No offense—I would love to help maintain these projects, actually! It's just that there are a couple of problems:

A) I don't know the Java 3D codebase very well. So I probably don't fulfill your "know the beast" requirement. I've been in the position of sole maintainer of a project where my technical knowledge is insufficient even to intelligently review PRs. It's not a position I want to volunteer for again...

B) I am maintaining too many projects already. I know you said not to talk to you about time ;-), but my time budget for this is insanely limited.

All of that said, I'd be willing to be a *co*maintainer, with the understanding that similar to hharrison, my availability might be highly variable. But I'd be happy to help review changes related to my areas of knowledge, such as Maven.

Anyway, from gouessej's posts, it sounds like he is willing and able to step in and fill in the maintainership gap here, which is awesome.

ctrueden wrote

Anyway, from gouessej's posts, it sounds like he is willing and able to step in and fill in the maintainership gap here, which is awesome.

I just do what has to be done. Harvey, Emmanuel and August have a far better knowledge of Java3D than me. You can consider that I'm the new maintainer of this engine. I don't promise any new features but I'll go on maintaining it.

ctrueden wrote

I'll file PRs for those as soon as the PRs for the circular dependency are merged.

The Maven PRs for java3d-core and java3d-utils are now filed:

* https://github.com/gouessej/java3d-core/pull/2
* https://github.com/gouessej/java3d-utils/pull/2