Java3D in batch mode

What are some options for "batch" processing in a "no screen attached" situation. We have Linux server with with a new Nvidia graphics card and want to do Java3D rendering to an offscreen Canvas3D from an automated script. There's are a several discussions related to "headless" mode, but it's not clear what that means exactly.

Hi,

Strict Senso, Java3D's offscreen rendering allows you to create 3D images in a Canvas3D not attached to a visible AWT/Swing gadget. It is, you just create an offscreen Canvas3D and it will generate 3D for you. Extend it to create your own screen capture and you can export and jpg, png, etc.

Hi,

But how can that work, Java3D's Canvas3D extends awt.Canvas?

Thomas,
Sorry, I do intend to respond to your question, I've just been away from a computer for a while, I will answer this as it's a great question and for example off screen batch mode is being used by sweethome3d.com to create videos, which is super cool.

Phil,

Time away from the computer is a good thing. Enjoy. I appreciate all the help from everyone on the list, you and Julien in particular.

Tom

sweethome3d is pretty awesome, but bypassing graphics acceleration to an object file format, and software ray tracing would likely just be too slow for our operational requirement.

Would just installing a local X11 server on the target machine work? It seems that's where the awt.HeadlessException comes from?

Has anyone used VirtualGL with Java3D in this situation?

Unfortunately I'm no Linux server expert so I really can't help with how to get a GraphicsEnvironment working with teh screen unplugged.

These days I mostly work with my adaptation of Java3D on Android that has no AWT in it at all, and allows me to simply create an OpenGL surface and give it to Java3D to work on.
new Canvas3D(GLWindow.create(new GLCapabilities(null))).

The examples for Java3D 1.6 have an offscreen example .
However it has 2 issues; 1 it’s not very dynamic so it’s hard to see what it can do just from the demo  and ; 2 most of the code in it is focused on coordinating an on screen canvas and an off screen canvas and a raster between them. With a lot of work you could turn this into a sort of “security camera in a 3d shooter” type demo, but it distracts from the power of the off screen rendering.
I would mention that in your case (of not having a screen plugged into your GPU card) I would expect GraphicsDevice.getBestConfiguration to work fine. GraphicsDevice.getBestConfiguration eventually through a chain of calls arrives at JoglPipeline.getBestConfiguration which calls new Frame() and there is a chance that any given configuration may be unable to instantiate that Frame, and hence the init code will fail. In that case you might consider using Java3D 1.7 with the Android (no-AWT) adaptation I made.

You need to manually invoke Canvas3D.renderOffScreenBuffer(); and optionally Canvas3D.waitForOffScreenRendering(). But the scene graph can be updated and used as usual and you can do anything you like with the image you give to Canvas3D.setOffScreenBuffer including (for example) saving multiple images as a video like SweetHome3D  does, using Java3D for the lower quality renders and ray tracing for the higher.

Below I’ve written a very small example of what can be easily and quickly done with the off screen canvas. It requires the examples for Java3D 1.6 because I use the mildly interesting SphereMotion scene, though if you like you can just grab that one file .
For brevity this code has had the license removed, but please note it is derived from the OffScreenTest of the Java3D examples and is covered by that license.

Each time you click the 3D image it will take a copy and display it in the label below it
 
package _testcase;

import java.awt.GraphicsConfiguration;
import java.awt.GraphicsEnvironment;
import java.awt.event.MouseAdapter;
import java.awt.event.MouseEvent;
import java.awt.image.BufferedImage;

import javax.media.j3d.BranchGroup;
import javax.media.j3d.Canvas3D;
import javax.media.j3d.GraphicsConfigTemplate3D;
import javax.media.j3d.ImageComponent;
import javax.media.j3d.ImageComponent2D;
import javax.media.j3d.Screen3D;
import javax.media.j3d.View;
import javax.swing.ImageIcon;
import javax.swing.JLabel;

import com.sun.j3d.utils.universe.SimpleUniverse;

import org.jdesktop.j3d.examples.sphere_motion.SphereMotion;

public class OffScreen extends javax.swing.JFrame {

        private SimpleUniverse univ = null;
        private BranchGroup scene = null;

        private javax.swing.JPanel drawingPanel;

        private Canvas3D createOnScreenCanvasAndUniverse() {
                // Get the preferred graphics configuration for the default screen
                GraphicsConfiguration config = SimpleUniverse.getPreferredConfiguration();

                // Create a Canvas3D using the preferred configuration
                Canvas3D onScrCanvas = new Canvas3D(config, false);

                // Create simple universe with view branch
                univ = new SimpleUniverse(onScrCanvas);

                // This will move the ViewPlatform back a bit so the
                // objects in the scene can be viewed.
                univ.getViewingPlatform().setNominalViewingTransform();

                // Ensure at least 5 msec per frame (i.e., < 200Hz)
                univ.getViewer().getView().setMinimumFrameCycleTime(5);

                return onScrCanvas;
        }

        private OffScreenCanvas3D createOffScreenCanvas() {
                // request an offscreen Canvas3D with a single buffer configuration
                GraphicsConfigTemplate3D template = new GraphicsConfigTemplate3D();
                template.setDoubleBuffer(GraphicsConfigTemplate3D.UNNECESSARY);
                GraphicsConfiguration gc = GraphicsEnvironment.getLocalGraphicsEnvironment().getDefaultScreenDevice()
                                .getBestConfiguration(template);

                // Create a offscreen Canvas3D using the single buffer configuration.
                OffScreenCanvas3D offScrCanvas = new OffScreenCanvas3D(gc, true);

                return offScrCanvas;
        }

        public OffScreen() {
                initComponents();

                // Create the content branch and add it to the universe
                scene = new BranchGroup();
                scene.addChild(new SphereMotion(new String[0]).createSceneGraph());

                // Create an OnScreenCanvas3D and SimpleUniverse; add canvas to drawing panel
                Canvas3D onScreenCanvas = createOnScreenCanvasAndUniverse();
                drawingPanel.add(onScreenCanvas, java.awt.BorderLayout.CENTER);

                // build a JLabel to display the image buffer
                ImageIcon output = new ImageIcon();
                JLabel outputJL = new JLabel(output);
                drawingPanel.add(outputJL, java.awt.BorderLayout.SOUTH);

                // Create an OffScreenCanvas3D
                OffScreenCanvas3D offScreenCanvas = createOffScreenCanvas();
                output.setImage(offScreenCanvas.bImage);

                // set the offscreen to match the onscreen
                Screen3D sOn = onScreenCanvas.getScreen3D();
                Screen3D sOff = offScreenCanvas.getScreen3D();
                sOff.setSize(sOn.getSize());
                sOff.setPhysicalScreenWidth(sOn.getPhysicalScreenWidth());
                sOff.setPhysicalScreenHeight(sOn.getPhysicalScreenHeight());

                //Because we have both on screen and off screen we need to attach the offscreen canvas to the universe view
                View view = univ.getViewer().getView();
                view.addCanvas3D(offScreenCanvas);

                //************************ on a click update the image shown in the JLabel
                onScreenCanvas.addMouseListener(new MouseAdapter() {
                        @Override
                        public void mouseClicked(MouseEvent e) {
                                offScreenCanvas.renderOffScreenBuffer();
                                offScreenCanvas.waitForOffScreenRendering();
                                // force the image to update
                                outputJL.setIcon(output);
                                outputJL.revalidate();
                                outputJL.repaint();
                        }
                });

                univ.addBranchGraph(scene);
        }

        private void initComponents() {
                drawingPanel = new javax.swing.JPanel();
                setDefaultCloseOperation(javax.swing.WindowConstants.EXIT_ON_CLOSE);
                drawingPanel.setLayout(new java.awt.BorderLayout());
                drawingPanel.setPreferredSize(new java.awt.Dimension(500, 500));
                getContentPane().add(drawingPanel, java.awt.BorderLayout.CENTER);
                pack();
        }

        public static void main(String args[]) {
                System.setProperty("sun.awt.noerasebackground", "true");
                java.awt.EventQueue.invokeLater(new Runnable() {
                        public void run() {
                                new OffScreen().setVisible(true);
                        }
                });
        }

        class OffScreenCanvas3D extends Canvas3D {
                BufferedImage bImage;
                ImageComponent2D buffer;

                public OffScreenCanvas3D(GraphicsConfiguration gconfig, boolean offscreenflag) {
                        super(gconfig, offscreenflag);
                        bImage = new BufferedImage(200, 200, BufferedImage.TYPE_INT_RGB);

                        buffer = new ImageComponent2D(ImageComponent.FORMAT_RGB, bImage, true, false); // notice yup = false because we are rendering onto an ImageIcon
                        buffer.setCapability(ImageComponent2D.ALLOW_IMAGE_READ);

                        this.setOffScreenBuffer(buffer);
                }
        }
}

 

philjord wrote

Unfortunately I'm no Linux server expert so I really can't help with how to get a GraphicsEnvironment working with teh screen unplugged.

I wonder if it's possible to serialize an interactive GraphicsEnvironment, or otherwise fake it?

These days I mostly work with my adaptation of Java3D on Android that has no AWT in it at all, and allows me to simply create an OpenGL surface and give it to Java3D to work on.
new Canvas3D(GLWindow.create(new GLCapabilities(null))).

Is this a new ctr for Canvas3D in Java3D 1.7? GLWindow.create() doesn't return a GraphicsConfiguration does it?

I would mention that in your case (of not having a screen plugged into your GPU card) I would expect GraphicsDevice.getBestConfiguration to work fine. GraphicsDevice.getBestConfiguration eventually through a chain of calls arrives at JoglPipeline.getBestConfiguration which calls new Frame() and there is a chance that any given configuration may be unable to instantiate that Frame, and hence the init code will fail. In that case you might consider using Java3D 1.7 with the Android (no-AWT) adaptation I made.

Is this the ctr above you're talking about?

You need to manually invoke Canvas3D.renderOffScreenBuffer(); and optionally Canvas3D.waitForOffScreenRendering(). But the scene graph can be updated and used as usual and you can do anything you like with the image you give to Canvas3D.setOffScreenBuffer including (for example) saving multiple images as a video like SweetHome3D  does, using Java3D for the lower quality renders and ray tracing for the higher.

Below I’ve written a very small example of what can be easily and quickly done with the off screen canvas.

I apologize for not explaining my situation more carefully, we can already work with the offscreen facilities that Java3D provides, so I made you write some code you didn't have to. Though this might be a good starting point for a unit test to experiment with.

Can we talk about how you've been able to bypass AWT in Java3D 1.7 for offscreen mode for non Android? If you have time.

Tom

As far as I understand, he added the possibility to create a Canvas3D from a NEWT GLWindow. However, as long as org.jogamp.java3d.Canvas3D extends java.awt.Canvas, it will be a problem under Android. I assume that Android support requires much more changes.

I don't need Android. I need as Philjord suggests: "create an OpenGL surface for Java3D to work on w/o AWT" I'm only supposing that OpenGL surface means a substitute for awt.Canvas->Canvas3D

Tom,

Sorry for misunderstanding I thought you were able to create the Canvas3D and were just discussing the off screen options.

I see now that you have a GPU card but the OS is not letting the windowed elements instantiate for some reason.

I presume you've knocked together a test app and run it and got some sort of exception thrown, can you provide that?

Is it that anything that extends java.awt.Component will not run at all? You can't even instantiate and get doNotify() called on a trivial JPanel?

Or are you getting something like a headless exception from Java3D; which (if the AWT part works) might just be an overly enthusiastic sanity test in Java3D we can disable.

If it is the case that you can't get anything in java.awt.* to run at all then we could look at my extension. In order for it to be a viable option you'd need to confirm that GLWindow.create(new GLCapabilities(null)) from Jogl 2.3.2 will return a working GLWindow, if so we could discuss my extension further.

Depending on how you are using Java3D you may or may not be familiar with the underlying Jogl system, if you aren't I can point you to some simple test to confirm it's status on your machine.


And finally no worries about the code snippet, it actually took only a few moments to write. It took me a long time to answer your post as I went down a rabbit hole trying to understand how an on-screen and off-screen Canvas3D can interoperate together with a Raster in a manner that gets the raster updated without any race conditions (I can report that it is complex).
Phil.

philjord wrote

Tom,

Sorry for misunderstanding I thought you were able to create the Canvas3D and were just discussing the off screen options.

I see now that you have a GPU card but the OS is not letting the windowed elements instantiate for some reason.

I presume you've knocked together a test app and run it and got some sort of exception thrown, can you provide that?

Is it that anything that extends java.awt.Component will not run at all? You can't even instantiate and get doNotify() called on a trivial JPanel?

Or are you getting something like a headless exception from Java3D; which (if the AWT part works) might just be an overly enthusiastic sanity test in Java3D we can disable.

Our system can run interactive or w/o instantiating any AWT windows/component. My guess is that Java3D just can't work w/o a physical display plugged into a port on the machine. Maybe this is a AWT limitation? I modified your code to only try to create an offscreen Canvas3D:

Exception in thread "main" java.awt.HeadlessException:
No X11 DISPLAY variable was set, but this program performed an operation
which requires it.
     at
sun.java2d.HeadlessGraphicsEnvironment.getDefaultScreenDevice(HeadlessGraphicsEnvironment.java:77)
     at MyOffScreen.createOffScreenCanvas(MyOffScreen.java:51)
     at MyOffScreen.<init>(MyOffScreen.java:77)
     at MyOffScreen.main(MyOffScreen.java:118)

philjord wrote

If it is the case that you can't get anything in java.awt.* to run at all then we could look at my extension. In order for it to be a viable option you'd need to confirm that GLWindow.create(new GLCapabilities(null)) from Jogl 2.3.2 will return a working GLWindow, if so we could discuss my extension further.

Depending on how you are using Java3D you may or may not be familiar with the underlying Jogl system, if you aren't I can point you to some simple test to confirm it's status on your machine.

Can you provide this test code? I will try to create a GLWindow as you suggest above when I'm back in the office.

philjord wrote

And finally no worries about the code snippet, it actually took only a few moments to write. It took me a long time to answer your post as I went down a rabbit hole trying to understand how an on-screen and off-screen Canvas3D can interoperate together with a Raster in a manner that gets the raster updated without any race conditions (I can report that it is complex).
Phil.

Kudos finding your way out! If memory serves we invoke renderOffscreenBuffer and grab the image in Canvas3D.postSwap.

Tom

Phil,

GLWindow glWindow = GLWindow.create(new GLCapabilities(null));

throws this:

Caught handled GLException: EGLGLXDrawableFactory - Could not initialize shared resources for EGLGraphicsDevice[type .egl, v1.5.0, connection nil, unitID 0, handle 0x7f8de40012d0, owner true, ResourceToolkitLock[obj 0x5fcac675, isOwner true, <7f2266f1, 352f15fb>[count 1, qsz 0, owner <main-SharedResourceRunner>]]] on thread main-SharedResourceRunner
    [0]: jogamp.opengl.egl.EGLDrawableFactory$SharedResourceImplementation.createSharedResource(EGLDrawableFactory.java:518)
    [1]: jogamp.opengl.SharedResourceRunner.run(SharedResourceRunner.java:353)
    [2]: java.lang.Thread.run(Thread.java:748)
Caused[0] by GLException: Graphics configuration failed [direct caps, eglGetConfig/chooser and fixed-caps(1-3)] on thread main-SharedResourceRunner
    [0]: jogamp.opengl.egl.EGLGraphicsConfigurationFactory.chooseGraphicsConfigurationStatic(EGLGraphicsConfigurationFactory.java:317)
    [1]: jogamp.opengl.egl.EGLDrawableFactory.evalConfig(EGLDrawableFactory.java:1060)
    [2]: jogamp.opengl.egl.EGLDrawableFactory.createSurfacelessImpl(EGLDrawableFactory.java:1088)
    [3]: jogamp.opengl.egl.EGLDrawableFactory$SharedResourceImplementation.mapAvailableEGLESConfig(EGLDrawableFactory.java:705)
    [4]: jogamp.opengl.egl.EGLDrawableFactory$SharedResourceImplementation.createEGLSharedResourceImpl(EGLDrawableFactory.java:613)
    [5]: jogamp.opengl.egl.EGLDrawableFactory$SharedResourceImplementation.createSharedResource(EGLDrawableFactory.java:516)
    [6]: jogamp.opengl.SharedResourceRunner.run(SharedResourceRunner.java:353)
    [7]: java.lang.Thread.run(Thread.java:748)
Exception in thread "main" com.jogamp.nativewindow.NativeWindowException: X11Util.Display: Unable to create a display(nil) connection. Thread main
        at jogamp.nativewindow.x11.X11Util.openDisplay(X11Util.java:453)
        at jogamp.opengl.x11.glx.X11GLXDrawableFactory$SharedResourceImplementation.isDeviceSupported(X11GLXDrawableFactory.java:230)
        at jogamp.opengl.SharedResourceRunner.getOrCreateShared(SharedResourceRunner.java:224)
        at jogamp.opengl.x11.glx.X11GLXDrawableFactory.getOrCreateSharedResourceImpl(X11GLXDrawableFactory.java:371)
        at jogamp.opengl.x11.glx.X11GLXDrawableFactory.getOrCreateSharedResourceImpl(X11GLXDrawableFactory.java:81)
        at jogamp.opengl.GLDrawableFactoryImpl.getOrCreateSharedResource(GLDrawableFactoryImpl.java:187)
        at jogamp.opengl.GLDrawableFactoryImpl.createSharedResourceImpl(GLDrawableFactoryImpl.java:216)
        at com.jogamp.opengl.GLDrawableFactory.createSharedResource(GLDrawableFactory.java:381)
        at com.jogamp.opengl.GLProfile.initProfilesForDeviceCritical(GLProfile.java:1916)
        at com.jogamp.opengl.GLProfile.initProfilesForDevice(GLProfile.java:1875)
        at com.jogamp.opengl.GLProfile.initProfilesForDefaultDevices(GLProfile.java:1843)
        at com.jogamp.opengl.GLProfile.access$000(GLProfile.java:80)
        at com.jogamp.opengl.GLProfile$1.run(GLProfile.java:230)
        at java.security.AccessController.doPrivileged(Native Method)
        at com.jogamp.opengl.GLProfile.initSingleton(GLProfile.java:216)
        at com.jogamp.opengl.GLProfile.getDefaultDevice(GLProfile.java:2027)
        at com.jogamp.opengl.GLCapabilities.<init>(GLCapabilities.java:84)

Phil,

Any thoughts? Can we just say that Java3D must have a local X11 server with a physical display monitor plugged in for both onscreen and offscreen rendering? Because this is a requirement of java.awt? I appreciate your help, but I think the time has come to punt on this. I'm required to bring this to a conclusion very soon.

Tom

Unfortunately if the Jogl isn't able to set up a surface then we are a long way from making Java3D work, and efforts to remove AWT or suppress warning aren't going to help.

The test line you ran is about as simple as it gets for Jogl so it's pretty definitive.

Someone else on the forum may provide help to get an X11 setup such that Jogl can work with it, but my experience is mainly with Windows where the windowing environment is almost always running regardless of the use of the machine.

Good luck with whatever solution you come up with (even if it's only plugging in a cheap monitor).

Could I be a real pain and ask you to put a final wrap up note on this forum at some point in the future so others can see what your final outcome was to assist them in case they are going down the same path? Even if you give up on off screen, or plugin a monitor or do something else.

Thanks,
Phil.