Brainstorming Vulkan Integration ...

Brainstorming Vulkan Integration ...
https://jogamp.org/wiki/index.php?title=SW_Tracking_Report_Feature_Objectives_Overview#Vulkan

Reason I reiterate this were a few mentions in recent discussions.

While I am willing to jump on this task, it probably would need
some weeks work, required to be funded. I know ...
But regardless, let me continue with the 'storm' :)

+++

Currently we have 3 basic building blocks within JOGL
A - Windowing System
B - GPU API (OpenGL)
C - Support (EventListener, PMVMatrix, ...)

Vulkan would enhance (B), adding one more GPU API,
while it shall reuse (A) + (C).

Nothing magic, just tedious work
with a requirement for good abstraction.

Higher top-level tools like GraphUI could then easily implement their
GPU depended part using Vulkan.

Perhaps some opengl package names within (A) + (C) should then be renamed
to 'cg' (computer graphics) like 'math' .. however, that is just lipstick
and maybe done on the go.

(edited)

Hello

It would be nice to document how to reuse shaders between Vulkan and WebGPU as the support of SPIR-V in WebGPU is only transitional.

Hi Sven,

I am not sure Vulkan support is worth the effort. It is probably a tedious task and you can't do other JOGL improvements while doing this.

Vulkan has mainly been pulled by the gaming industry which performance requirements led toward more specialized and fine grained interface to the rendering hardware. The result is that Vulkan API is very complex and verbose.

I do not know video games written in Java. I think the Java ecosystem relating to 3D rendering is mainly in the field of scientific visualization. This area does not require high performance. Software vendor using Java and JOGL mainly want to have descent performance and ability to build 3D visualization software with little effort (java to write once/run everywhere, JOGL to avoid diving in CGL/WGL/GLX). I fear that the current users of JOGL would not want to learn and use Vulkan, unless OpenGL disappear from earth - but I think this won't happen.  

If enabling Vulkan is consider valuable, it would probably not be enough per se. My understanding is that Vulkan and Metal are very similar but that Vulkan does not exists on macOS. So if JOGL supports Vulkan, it should support Metal at the same time.

What do you think?

I'd like to know Sven's opinion me too for sure.

At first, some work related to Vulkan would benefit to the possible support of other APIs than OpenGL but I admit that our time is precious. I know a little bit Vulkan and WebGPU; if Sven decides to add Vulkan support, I'll have to find some time to contribute to some aspects including the automatic generation of Java methods by using GlueGen and some possible improvements in this area. I would like to avoid causing another development hiatus, we're a team after all and a more balanced way of sharing the work to do would limit the risk of slowing down improvements in JOGL.

Secondly, yes the Vulkan API is drastically much more complicated than OpenGL. OpenGL and OpenGL-ES won't suddenly disappear tomorrow but on the long term, OpenGL is a dead end, it's not only a matter of gaming or scientific visualization, WebGL will become unsupported under OS X and OpenGL will follow the same path too, sticking to OpenGL only isn't future proof anyway.

Thirdly, it's not a secret, the Java ecosystem isn't in a very good health, we don't have the luxury of having separate APIs for gaming and for scientific visualization even though Jzy3D is an excellent counterexample. I disagree about the need of high performance in scientific visualization, it's a matter of particular use cases, maybe your affirmation is true most of the time but I had at least one position in my career in which I saw that performance was a major concern in scientific visualization and there were some similarities between the unstructured meshes manipulated at work and ... the museum displayed within T.U.E.R which is a game. By the way, it explains why I'll have to use a library almost exclusively used in scientific visualization through Gephi in a game editor. There isn't a big thick wall between gaming and scientific visualization but there are often different needs.

Fourthly, most games written for Android are written in Java, which isn't true for games written for desktop environments. GNU Linux is slowly becoming a viable option in mobile environments; if we focus only on the needs of scientific visualization, we'll miss the train and we'll leave a big avenue for our major competitor which is already in a comfortable position in gaming. Please note that there are only a very few requests for improvement in JOGL related to gaming, there's mainly one about an important improvement in mouse management in NEWT.

Fifthly, you can use Vulkan under OS X with MoltenVK and similar solutions.

To sum up, in my humble opinion, Vulkan and very low-level modern rendering APIs like WebGPU aren't interesting only for gaming, we'll have to pick one of them anyway. Metal is a no go for me because it works only under OS X, WebGPU would be a better candidate but it wasn't born yet. Anyway, many Java APIs using a Java binding for OpenGL are already unable to work with a fully shader based pipeline, this problem already exists right now, you'll have to choose between consuming a lot of time for few end users with drivers supporting only the forward compatible mode or using the bare-to-the-metal API to rewrite a lot of code if and only if you don't have the time to do both. As I'm alone on JogAmp's Ardor3D Continuation, I will have to make a decision and you'll have to make your own decision for Jzy3D except if we work together to render Jzy3D by using a tiny subset of my engine which would be beneficial to you and me.

Dear Martin, dear Julien, et al,

now that is a refreshing discussion, thank you!

Forgive me my potential rambling in between, I also will not proof read my text,
so quality and mileage may vary - my apologies.


+++

When I write about OpenGL from here on, I explicitly mean core, shader based programmable only.

IMHO, OpenGL will stay as-is and it gives its decent level of 'bare metal' abstraction as we have it.
It is stable and hence very valuable. This even more so for the embedded field.
In the days ahead, it might be implemented using Vulkan, a wrapper. Which is OK.

Vulkan is a beast of details and hence _seems_ to distract from the goals. A similar perception like
when OpenGL ES2 arrived ;-)
In its inception they also claimed to not use it, but have it being used by other libraries.
Goals were a more simpler way for GPU vendor implementations, drop states and allow the more fine grained resource management by the user.
Up to this point, OpenGL can be used as well in a similar way .. using shared context and uniforms, attributes and sync points.

New detailed features arrived in Vulkan and it became a one for all API: Graphics, Video, Processing.
(Yes, still a PIA lol)

Bottom line, both, OpenGL and Vulkan, expose the hardware path to software, in a very detailed way and need some support libraries to put it to good use and allow the developer to focus on their actual goals.

+++

Re: Vulkan may or may not be supported on Apple's device.
Re: OpenGL/Vulkan versus the rest of the bunch

(the rambling part)

I simply don't care personally in a way that I also don't care about WebGPU (at this time).
Apple chose to go ahead with their 'stolen' ATI/AMD pre-metal tech
and not to re-consolidate with the consequential successor which is Vulkan.

I know they have a lot of users, which also use JogAmp (thank you).
But on a personal level, Metal really makes no sense these days.
Its more a political Apple decision to separate things as usual.

On a commercial level, I would even integrate Metal to JogAmp,
especially these days as we have to be flexible (or better: whores of the markets lol).
But it makes no sense.
Hence OpenGL is a good and viable compromise esp on this platform.

Now WebGPU is a different beast, sort of trying to consolidate the APIs
and hence compromising. Sort of what we offer ;-)
However, it is driven by the browser guys AFAIK and heck have they
ruined the desktop in the last decades to a result, where even desktop applications
are hacked together by JavaScript 'solutions' :)
Funny .. until its no more.
And we talk about high performance Java vs C++? Hilarious.
But maybe I mistake WebGPU for WebGL .. no, not really.

OpenGL and Vulkan at least come from an older group of graphics folks
(sponsored by GPU vendors and also the software folks) who still consider the
bare metal desktop and embedded devices as a target platform.
IMHO this is as good as it gets for a 'vendor neutral' GPU plug.

Abstractions, managed code and helpers .. well, until these days, we have written them ourselves.

In a bigger project (even used in like Graph), we actually assemble or write our shader ourselves
in the library before usage - so I would think any additional redirection could be distracting (bugs).
But it surely is a good way to have more work.

OK, you might think let's use an abstraction layer around Vulkan, Metal etc
and have it all working at no costs.
Maybe, but when the underling is more different than the design goal, there will be costs.
All of this sort of fights the whole purpose of having a vendor neutral GPU API like OpenGL and Vulkan.
Yes, it helps to deal with the outliers like Direct 3D (really still in use?) and Metal ..
so more of a political/commercial aspect ;-)

+++

Games, science, .. indeed, it doesn't matter. Both roads follow the path of a (concept) paper (and design), iterative developer cycles and so forth. If massive data is involved, algorithms better help the poor GPU ;-)

+++

A new binding these days from a Khronos Group API IMHO should be done differently using their xml description instead of parsing the C header. The latter has less information like loss of 'read only' etc.

So we would have to just read the xml tables and then use GlueGen's output 'driver'.

This portion of the work, while interesting, is tedious indeed and the challenge will be (again)
to make the different Vulkan version's OO overlapping interfaces - besides perhaps separation for
the different fields (core, video, processing, ...).

After that would have been done (and we may cheat and steal for ideas in a legal manner) ..
the real work would begin: Glueing the whole API together once again ... :)

Goal IMHO would be to use Vulkan or OpenGL besides the basic set of our support API.

+++

Java's health for the front-end (visuals, UI etc.).

Its a quite funny time we live in. The JVM these days is the best ever hotspot compiling beast
and others (Azul etc) are even trying to further this with llvm.
So the virtual machine is an excellent platform - if you like to use them.

My late C++17/20 'rebirth' gave me much more language expressiveness back,
comparing C++ to Java - the latter is rather lame :)
However, if using the language in a higher level use case, nobody really cares IMHO.

Compared to Python et al, Java+JVM is surely the winner.
I even could say, JVM wins against a Docker or whatever virtualization beast, its build-in.

The real reason Java lost attention in UI and desktop is politics and "Oracle's ways".
- allowed the browser group to abandon the applet and hence JNLP.
- stopped JavaFX.
- told the world: We don't care about front end, but here you are ..

Hence some people work hard to provide means for the front-end, OpenJFX and so forth,
even us little shrimps here ;-)

Now .. to provide a company with a viable solution,
they not only need a good solution - which we may have,
but also need to trust the lifecycle and available support.
The latter is the real issue IMHO, a chicken and egg problem.

They won't give me or us money for R&D since we can't provide a 10 year timeline ahead
and b/c we have no money.
Result: We still have no money and still can't provide a good timeline :)
Its really not a technical issue.
I had a few chats in the embedded / automotive field in this regard.

Technically I would claim we have the means, but resource wise we are the poor.

+++

Time management etc .. is an issue of course, especially if I would jump to another project again etc.

Purpose of this discussion is to handle the ideas & impressions to perhaps SLOWLY sorting things out BEFORE even starting such work in the FUTURE.

Of course, first we need to keep our reason of existence alive, a working solution using OpenGL, OpenAL, OpenCL, FFMpeg etc. next to our own support stuff working in harmony.

Only if time and resources (funding & contracts) will allow, an expansion to other APIs is viable.
It would also make no sense for me to jump into the Vulkan Khronos-XML-GlueGen pond
and abandon showing use cases like GraphUI, while not having the funding.
Because funding also implies a better (commercial) user base which we wouldn't need to win anymore ;-)
The chicken and egg problem.

Not being afraid of the wind, let's see what the future has to see for us and the project,
as well as for us personally paid-work wise and otherwise.
Both would need to come together to resolve the resource problem we have, I know.
Years ago I was hopeful to have more funded and worked on this promise and goal, but it failed miserably.
Sorry for that!

Maybe we should create a bug report for that:
https://jogamp.org/bugzilla/show_bug.cgi?id=1137#c6

I plan to start working a bit on Vulkan before the end of the month.