Custom Raster

Dear JOGL Team,

I'm doing some offscreen rendering. An instance of the IntegerInterleavedRaster works, but if I use an extended class the rendering is not done anymore. I suppose the class of the given raster instance is used as identifier at some point.

I would like to register my own renderer. How should I proceed?

Thanks.

Regards,
JP

Are you talking about sun.awt.image.IntegerInterleavedRaster?

JeanPhi wrote

Dear JOGL Team,

I'm doing some offscreen rendering. An instance of the IntegerInterleavedRaster works, but if I use an extended class the rendering is not done anymore. I suppose the class of the given raster instance is used as identifier at some point.

IntegerInterleavedRaster is unrelated to JOGL, you may want to ask on the OpenJDK java2d mainlinglist that maintains the AWT API.
http://www.docjar.com/docs/api/sun/awt/image/IntegerInterleavedRaster.html
http://openjdk.java.net/groups/2d/

JeanPhi wrote

I would like to register my own renderer. How should I proceed?

Thanks.

Regards,
JP

I suggest you to use the programmable GPU instead of AWT/CPU and use a GPU fragment shader in order to have your custom renderer.

Try look at the jogl testcases and demos.
A book on modern OpenGL shaders is also a good start on how to write your own rasterizer/renderer using the GPU.
http://www.arcsynthesis.org/gltut/Basics/Tut01%20Following%20the%20Data.html
https://github.com/elect86/modern-jogl-examples/tree/master/modern-jogl-examples/src/tut01

Cheers
Xerxes

Yes, I'm talking about the sun.awt.image.IntegerInterleavedRaster. In fact I'm trying to intercept Raster updates to forward them to a JavaFX Canvas. I will take a look to custom GPU stuff, but this is a temporary solution so I would like to keep it as simple as possible.

The story starts with Java3D which uses an ImageComponent2D as offscreen buffer. This buffer contains a BufferedImage which itself contains a raster. If I use an instance of IntegerInterleavedRaster the rendering is done without any problem. But if I provide even a subclass of IntegerInterleavedRaster, then the rendering is not performed.

It seems the class type of the raster is a key of the system. I would like to add my own key/raster. Is there a way to do so?

JP

Why not simply putting the Java3D canvas into a SwingNode?

Because it doesn't work :)

JavaFX only allows lightweight Swing components inside a SwingNode. I tried JCanvas3D but the CPU load is huge not to mention the awful display tearing.

I tried to follow what's going on in Java3D and my best bet is JOGL to handle rasters "by type". I hope all the types are not hard coded and there is somewhere where I could register mine.

If I fail to intercept raster updates, then I will have to copy the buffer, which will be twice the work.

How and where are rasters identified in JOGL?

JeanPhi wrote

...
If I fail to intercept raster updates, then I will have to copy the buffer, which will be twice the work.

How and where are rasters identified in JOGL?

Remember the aim of JOGL is to allow Java application to access the GPU using hardware accelerated API's such as OpenGL. OpenGL Rasterization is done by code running inside the GPU thus its not possible to intercept GPU raster updates using java code running on the CPU.
Take a look at this diagram: https://www.opengl.org/wiki/Rendering_Pipeline_Overview
the programmer primes the OpenGL pipeline by providing the blue boxes (JOGL provides Java bindings to do this)
the GPU and its driver perform the yellow boxes running in parallel without interference from the CPU. (rasterization is done here by the GPU, JOGL simply wait for the GPU to complete, there is no API in JOGL or OpenGL to mess with the GPU while it do its work)
the end result is stored in the white boxes.. (JOGL provide Java API's to get the framebuffer etc)

http://www.youtube.com/watch?v=-P28LKWTzrI -  Mythbusters Demo GPU versus CPU

Thus you do not want to intercept the rasterization using a CPU because the rasterization is best done in parallel using the GPU.
Ideally you want OpenGL to paint directly using the right destination thus no copy shall be needed after rasterization.

JeanPhi wrote

Because it doesn't work :)

JavaFX only allows lightweight Swing components inside a SwingNode. I tried JCanvas3D but the CPU load is huge not to mention the awful display tearing.

Thank you for the feedback. The problem is that Java3D 1.6.0 is actively maintained but there is no plan to add new features into it (including OpenJFX/JavaFX interoperability). I think that you would get better performance with a JavaFX Java3D panel but writing such a panel wouldn't be trivial.

You are right, intecepting raster is not wished, but I need a temporary solution.

From Java3D I can do an offscreen render - note that Java3D uses JOGL. The rendering data will be stored in a raster (IntegerInterleavedRaster in my case). So, there is necessarily at least one method which is called in the raster to update its data :)

I would like my own IntegerInterleavedRaster extended class to be recognized. At which point the IntegerInterleavedRaster is identified and fed/updated?

I went further. The raster contains a DataBufferInt. It seems that JOGL writes the data directly inside the array contained in the DataBufferInt.

So I would have to copy those data to JavaFX :)

Am I right?

The Raster Java Object is never used directly by the JOGL Java3D pipeline,
instead the only thing that gets passed to JOGL is the image buffer inside
the Raster object. Take a look inside the Java3D sourcecode:
http://jogamp.org/git/?p=java3d/java3d-core.git;a=blob;f=src/classes/share/javax/media/j3d/JoglPipeline.java

Java3D asks the JOGL pipeline to do rasterization inside readRaster.. what
happens here is that JOGL then performs the copy from the framebuffer in
one go to the image buffer using glReadPixels  thus no method is called on
the Raster object to update the image buffer inside of it!
src/classes/share/javax/media/j3d/GraphicsContext3D.java:
Pipeline.getPipeline().readRaster(canvas3d.ctx,
src/classes/share/javax/media/j3d/Canvas3D.java:
Pipeline.getPipeline().textureFillRaster(ctx, texMinU, texMaxU, texMinV,
texMaxV,
src/classes/share/javax/media/j3d/Canvas3D.java:
Pipeline.getPipeline().executeRasterDepth(ctx, posX, posY, posZ,

https://www.opengl.org/sdk/docs/man4/html/glReadPixels.xhtml
http://www.opengl.org/archives/resources/faq/technical/rasterization.htm


2014-02-18 11:43 GMT+01:00 JeanPhi [via jogamp] <
ml-node+s762907n4031622h45@n3.nabble.com>:

> You are right, intecepting raster is not wished, but I need a temporary
> solution.
>
> From Java3D I can do an offscreen render - note that Java3D uses JOGL. The
> rendering data will be stored in a raster (IntegerInterleavedRaster in my
> case). So, there is necessarily at least one method which is called in the
> raster to update data :)
>
> I would like my own IntegerInterleavedRaster extended class to be
> recognized. At which point the IntegerInterleavedRaster is identified and
> fed/updated?
>
> ------------------------------
>  If you reply to this email, your message will be added to the discussion
> below:
> http://forum.jogamp.org/Custom-Raster-tp4031610p4031622.html
>  To start a new topic under jogl, email
> ml-node+s762907n782158h61@n3.nabble.com
> To unsubscribe from jogamp, click here<http://forum.jogamp.org/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=762907&code=eGVyeGVzQGd1ZGlubmEuY29tfDc2MjkwN3wtNTE5NjUwMzEw>
> .
> NAML<http://forum.jogamp.org/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>

Ok, good!

How to ensure is that the DataBufferInt is filled directly from the GPU data? Is the "filling" made by the GPU or the CPU?

JeanPhi wrote

How to ensure is that the DataBufferInt is filled directly from the GPU data?

This can be done by avoiding all copys from video ram to regular ram. Thus the solution is to not use any RAM buffer, like the DataBufferInt, at all for graphics. The GPU hardware can only read and write to Video-RAM, VRAM, memory, the VRAM memory can be initialized by the operating-system using the EGL API. JOGL uses EGL to initialize a native window that is then later used by OpenGL for rendering.

If you want high performance then you should make sure that all graphics API's are operating directly on the vram using native window handles and hardware accelerated API's. Thus a good start is to make OpenJFX able to expose its native window. JOGL and Java3D may reuse the OpenJFX native window handle for direct rendering without any VRAM <-> RAM data copys.

If you look inside the OpenJFX sourcecode you will notice that OpenJFX internally contains handwritten JNI code to initialize a native window using EGL and OpenGL ES 2 calls. OpenJFX currently do not contain any way to expose the native window for interoperability with other OpenGL applications and hardware accelerated API's. Access to the native window handles inside OpenJFX needs to get fixed if you want good interoperability between OpenJFX and Java3D.

JeanPhi wrote

Is the "filling" made by the GPU or the CPU?

glReadPixels is implemented by the GPU driver, on some hardware it is performed by the CPU, some hardware can use a DMA copy and some can use the GPU.
On embedded devices glReadPixels is expensive thus applications that use it will have limited framerate updated. the only solution is to avoid the copy and use direct vram rendering.

The workaround test basically works. The CPU is much lower than using JCanvas3D.

Now the bad news. The CPU is still very high (about 70% of a core for a 1500x1000 px scene @40FPS)

There are four methods which consume most of the CPU:
1. jogamp.opengl.gl4.GL4bcImpl.dispatch_glReadPixels1 (native)
2. javafx.scene.canvas.GraphicsContext.getBuffer()
3. jogamp.opengl.x11.glx.GLX.dispatch_glXMakeContextCurrent0 (native)
4. jogamp.opengl.gl4.GL4bcImpl.dispatch_glFinish1 (native)

I did not expect JOGL to consume so much :S

Maybe the problem is at Java3D level. When I'm in offscreen mode, there is no more automatic scene rendering. I have to call renderOffScreenBuffer() each time to render.

Is there a way to force the scene to be rendered automatically?

JP

J3DTest.java

Like this one?

Thank you very much.

Edit.: My bug report contains some pointers to working examples of interoperability between JavaFX and OpenGL based APIs:
https://jogamp.org/bugzilla/show_bug.cgi?id=607

Maybe it can help, especially the demo rendering OpenGL contents inside JavaFX without SwingNode of course (it uses a FBO).

Nice, I will read the JavaFX integration RFE :)

What is very strange is that Nasa World Wind (which uses OpenGL) succeeds in working inside JavaFX - AS IS! (no modification or JavaFX setting of any kind at all) - with no performance issue. So there is a way to do so!

Is there any Java3D/JOGL in JavaFX example which works?

Nasa World Wind uses JOGL.

There is a working example that uses another binding for OpenGL in the RFE but as I can't use OpenJFX/JavaFX at home, I haven't ported it to JOGL.

Edit.: NN WW seems to use SwingNode:
http://forum.worldwindcentral.com/showthread.php?36896-WorldWind-for-JavaFX&s=93dcf41770a3ea36b5c7f01373785f54&p=117506&viewfull=1#post117506

Didn't read that post. I just took the NWW and put it in a SwingNode. It works!

Did the same with Java3D and it failed... there must be a reason ;)