NEWTCanvasJFX Resize

Hi,

I am experimenting with NEWTCanvasJFX and see promising results however, I cannot get it to resize within a JavaFX window.  I have a stackpane which I add the NEWTCanvasJFX as a child in the controller initialization code.  The size for the NEWTCanvasJFX has to be set programmatically but the resize does not work.  The reshape method does not get called.

Here is the code:
package sample;

import com.jogamp.newt.javafx.NewtCanvasJFX;
import com.jogamp.newt.opengl.GLWindow;
import com.jogamp.opengl.GL4;
import com.jogamp.opengl.GLAutoDrawable;
import com.jogamp.opengl.GLCapabilities;
import com.jogamp.opengl.GLEventListener;
import com.jogamp.opengl.GLProfile;
import com.jogamp.opengl.awt.GLJPanel;
import com.jogamp.opengl.util.FPSAnimator;

import org.joml.Matrix4f;

import java.net.URL;
import java.nio.ByteBuffer;
import java.nio.ByteOrder;
import java.nio.FloatBuffer;
import java.nio.ShortBuffer;
import java.util.ResourceBundle;
import java.util.Scanner;

import javafx.application.Application;
import javafx.embed.swing.SwingNode;
import javafx.fxml.FXML;
import javafx.fxml.FXMLLoader;
import javafx.fxml.Initializable;
import javafx.scene.Parent;
import javafx.scene.Scene;
import javafx.scene.layout.StackPane;
import javafx.stage.Stage;

import static com.jogamp.opengl.GL.GL_ARRAY_BUFFER;
import static com.jogamp.opengl.GL.GL_BACK;
import static com.jogamp.opengl.GL.GL_BLEND;
import static com.jogamp.opengl.GL.GL_CCW;
import static com.jogamp.opengl.GL.GL_COLOR_BUFFER_BIT;
import static com.jogamp.opengl.GL.GL_CULL_FACE;
import static com.jogamp.opengl.GL.GL_DEPTH_BUFFER_BIT;
import static com.jogamp.opengl.GL.GL_DEPTH_TEST;
import static com.jogamp.opengl.GL.GL_ELEMENT_ARRAY_BUFFER;
import static com.jogamp.opengl.GL.GL_FLOAT;
import static com.jogamp.opengl.GL.GL_LEQUAL;
import static com.jogamp.opengl.GL.GL_SCISSOR_TEST;
import static com.jogamp.opengl.GL.GL_STATIC_DRAW;
import static com.jogamp.opengl.GL.GL_STENCIL_TEST;
import static com.jogamp.opengl.GL.GL_TRIANGLES;
import static com.jogamp.opengl.GL.GL_TRUE;
import static com.jogamp.opengl.GL.GL_UNSIGNED_SHORT;
import static com.jogamp.opengl.GL2ES2.GL_COMPILE_STATUS;
import static com.jogamp.opengl.GL2ES2.GL_FRAGMENT_SHADER;
import static com.jogamp.opengl.GL2ES2.GL_LINK_STATUS;
import static com.jogamp.opengl.GL2ES2.GL_VERTEX_SHADER;

public class Controller extends Application implements Initializable, GLEventListener
{
  @FXML
  private StackPane openGLPane;
  private long previousTime;
  private long frameCount;
  private int[] buffers;
  private int program;
  private float aspectRatio;
  private final float[] matrixArray = new float[16];
  private float angle;
  private NewtCanvasJFX glPanel;
  private boolean useExperimental = true;

  @Override
  public void start(Stage primaryStage) throws Exception
  {
    Parent root = FXMLLoader.load(getClass().getResource("sample.fxml"));
    primaryStage.setTitle("JavaFX OpenGL");
    primaryStage.setScene(new Scene(root, 300, 275));
    primaryStage.show();
  }

  public static void main(String[] args)
  {
    launch(args);
  }

  @Override
  public void initialize(URL location, ResourceBundle resources)
  {
    final GLProfile glProfile = GLProfile.getDefault();
    final GLCapabilities capabilities = new GLCapabilities(glProfile);

    if (useExperimental)
    {
      GLWindow glWindow = GLWindow.create(capabilities);

      glPanel = new NewtCanvasJFX(glWindow);
      glPanel.setWidth(300);
      glPanel.setHeight(300);
      openGLPane.getChildren().add(glPanel);

      System.out.println("openglpane resizable: " + openGLPane.isResizable());
      System.out.println("glPanel resizable: " + glPanel.isResizable());

      glWindow.addGLEventListener(this);

      final FPSAnimator animator = new FPSAnimator(glWindow, -1);
      animator.start();
    }
    else
    {
      final GLJPanel glPanel = new GLJPanel(capabilities);
      glPanel.addGLEventListener(this);

      final SwingNode swingNode = new SwingNode();
      swingNode.setContent(glPanel);
      openGLPane.getChildren().add(swingNode);

      final FPSAnimator animator = new FPSAnimator(glPanel, -1);
      animator.start();
    }
  }

  @Override
  public void init(GLAutoDrawable glAutoDrawable)
  {
    final GL4 gl = glAutoDrawable.getGL().getGL4();

    this.buffers = new int[2];
    gl.glGenBuffers(this.buffers.length, this.buffers, 0);

    final FloatBuffer positions =
        ByteBuffer.allocateDirect(8 * 3 * Float.BYTES).order(ByteOrder.nativeOrder()).asFloatBuffer();

    positions.put(.5f).put(.5f).put(.5f);    // 0
    positions.put(-.5f).put(.5f).put(.5f);   // 1
    positions.put(-.5f).put(-.5f).put(.5f);  // 2
    positions.put(.5f).put(-.5f).put(.5f);   // 3
    positions.put(.5f).put(.5f).put(-.5f);   // 4
    positions.put(-.5f).put(.5f).put(-.5f);  // 5
    positions.put(-.5f).put(-.5f).put(-.5f); // 6
    positions.put(.5f).put(-.5f).put(-.5f);  // 7

    positions.rewind();

    gl.glBindBuffer(GL_ARRAY_BUFFER, this.buffers[0]);
    gl.glBufferData(GL_ARRAY_BUFFER, positions.capacity() * Float.BYTES, positions, GL_STATIC_DRAW);

    final ShortBuffer indices =
        ByteBuffer.allocateDirect(12 * 3 * Short.BYTES).order(ByteOrder.nativeOrder()).asShortBuffer();

    indices.put((short) 0).put((short) 1).put((short) 2);
    indices.put((short) 0).put((short) 2).put((short) 3);
    indices.put((short) 4).put((short) 0).put((short) 3);
    indices.put((short) 4).put((short) 3).put((short) 7);
    indices.put((short) 4).put((short) 5).put((short) 1);
    indices.put((short) 4).put((short) 1).put((short) 0);
    indices.put((short) 1).put((short) 5).put((short) 6);
    indices.put((short) 1).put((short) 6).put((short) 2);
    indices.put((short) 3).put((short) 2).put((short) 6);
    indices.put((short) 3).put((short) 6).put((short) 7);
    indices.put((short) 5).put((short) 4).put((short) 7);
    indices.put((short) 5).put((short) 7).put((short) 6);

    indices.rewind();

    gl.glBindBuffer(GL_ELEMENT_ARRAY_BUFFER, this.buffers[1]);
    gl.glBufferData(GL_ELEMENT_ARRAY_BUFFER, indices.capacity() * Short.BYTES, indices, GL_STATIC_DRAW);

    final int vertexShader = compileShader(gl, GL_VERTEX_SHADER, loadCode("vertex.glsl"));
    final int fragmentShader = compileShader(gl, GL_FRAGMENT_SHADER, loadCode("fragment.glsl"));
    this.program = compileProgram(gl, vertexShader, fragmentShader);
  }

  private String loadCode(String resourceName)
  {
    try (final Scanner stream = new Scanner(this.getClass().getResourceAsStream("/" + resourceName)))
    {
      stream.useDelimiter("\\Z");

      return stream.next();
    }
  }

  @Override
  public void dispose(GLAutoDrawable glAutoDrawable)
  {

  }

  @Override
  public void display(GLAutoDrawable glAutoDrawable)
  {
    final long now = System.nanoTime();
    final long deltaTime = now - this.previousTime;
    ++this.frameCount;

    if (deltaTime >= 1_000_000_000L)
    {
      System.out.println("FPS: " + this.frameCount / (deltaTime / 1_000_000_000.0));
      this.frameCount = 0;
      this.previousTime = now;
    }

    final GL4 gl = glAutoDrawable.getGL().getGL4();

    gl.glEnable(GL_DEPTH_TEST);
    gl.glDepthFunc(GL_LEQUAL);
    gl.glDepthMask(true);
    gl.glClearDepthf(1);

    gl.glEnable(GL_CULL_FACE);
    gl.glCullFace(GL_BACK);
    gl.glFrontFace(GL_CCW);

    gl.glDisable(GL_BLEND);
    gl.glDisable(GL_SCISSOR_TEST);
    gl.glDisable(GL_STENCIL_TEST);

    gl.glColorMask(true, true, true, true);
    gl.glClearColor(0, 0, 0, 1);
    gl.glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);

    gl.glUseProgram(this.program);

    this.angle += Math.toRadians(1);

    new Matrix4f().setPerspective((float) Math.toRadians(60), this.aspectRatio, 0, 1000).mul(
        new Matrix4f().setLookAt(0, 0, 3, 0, 0, 0, 0, 1, 0)).mul(
        new Matrix4f().setRotationXYZ(this.angle, this.angle, 0)).get(this.matrixArray);

    gl.glUniformMatrix4fv(gl.glGetUniformLocation(program, "MVP"), 1, false, this.matrixArray, 0);
    gl.glUniform4f(gl.glGetUniformLocation(this.program, "color"), 0, 1, 0, 1);

    final int positionAttribute = gl.glGetAttribLocation(this.program, "position");

    gl.glEnableVertexAttribArray(positionAttribute);
    gl.glBindBuffer(GL_ARRAY_BUFFER, this.buffers[0]);
    gl.glVertexAttribPointer(positionAttribute, 3, GL_FLOAT, false, 0, 0);

    gl.glBindBuffer(GL_ELEMENT_ARRAY_BUFFER, this.buffers[1]);
    gl.glDrawElements(GL_TRIANGLES, 36, GL_UNSIGNED_SHORT, 0);
  }

  @Override
  public void reshape(GLAutoDrawable glAutoDrawable, int x, int y, int width, int height)
  {
    final GL4 gl = glAutoDrawable.getGL().getGL4();

    gl.glViewport(0, 0, width, height);
    this.aspectRatio = width / (float) height;
    System.out.println("reshape: " + width + " " + height);

    //glPanel.resize(width, height);
    if (glPanel != null)
    {
      glPanel.setWidth(width);
      glPanel.setHeight(height);
    }
  }

  private int compileShader(final GL4 gl, final int type, final String code)
  {
    int shader = gl.glCreateShader(type);

    if (shader != 0)
    {
      // add the source code to the shader and compile it
      gl.glShaderSource(shader, 1, new String[] {code}, new int[] {code.length()}, 0);
      gl.glCompileShader(shader);

      String statusString = "";
      int[] status = new int[1];

      gl.glGetShaderiv(shader, GL_COMPILE_STATUS, status, 0);

      if (status[0] != GL_TRUE)
      {
        int[] length = new int[1];
        byte[] statusBytes = new byte[1024];
        gl.glGetShaderInfoLog(shader, statusBytes.length, length, 0, statusBytes, 0);
        statusString = new String(statusBytes);
      }

      if (!statusString.isEmpty())
      {
        gl.glDeleteShader(shader);
        throw new RuntimeException("Could not compile shader: " + statusString + "\n" + code);
      }
    }

    return shader;
  }

  private int compileProgram(final GL4 gl, final int vertexShader, final int fragmentShader)
  {
    int programId = gl.glCreateProgram();

    if (programId != 0)
    {
      gl.glAttachShader(programId, vertexShader);
      gl.glAttachShader(programId, fragmentShader);

      gl.glLinkProgram(programId);

      gl.glUseProgram(programId);

      String programStatus = "";
      int[] status = new int[1];

      gl.glGetProgramiv(programId, GL_LINK_STATUS, status, 0);

      if (status[0] != GL_TRUE)
      {
        int[] length = new int[1];
        byte[] statusBytes = new byte[1024];
        gl.glGetShaderInfoLog(programId, statusBytes.length, length, 0, statusBytes, 0);
        programStatus = new String(statusBytes);
      }

      if (!programStatus.isEmpty())
      {
        gl.glDeleteProgram(programId);
        throw new RuntimeException("Could not link program: " + programStatus);
      }
    }

    return programId;
  }
}

FXML:
<?xml version="1.0" encoding="UTF-8"?>

<?import javafx.scene.control.Menu?>
<?import javafx.scene.control.MenuBar?>
<?import javafx.scene.control.MenuItem?>
<?import javafx.scene.layout.StackPane?>
<?import javafx.scene.layout.VBox?>

<VBox maxHeight="-Infinity" maxWidth="-Infinity" minHeight="-Infinity" minWidth="-Infinity" prefHeight="768.0" prefWidth="1024.0" xmlns="http://javafx.com/javafx/8.0.171" xmlns:fx="http://javafx.com/fxml/1" fx:controller="sample.Controller">
    <children>
        <MenuBar>
            <menus>
                <Menu mnemonicParsing="false" text="File">
                    <items>
                        <MenuItem mnemonicParsing="false" text="Close" />
                    </items>
                </Menu>
                <Menu mnemonicParsing="false" text="Edit">
                    <items>
                        <MenuItem mnemonicParsing="false" text="Delete" />
                    </items>
                </Menu>
                <Menu mnemonicParsing="false" text="Help">
                    <items>
                        <MenuItem mnemonicParsing="false" text="About" />
                    </items>
                </Menu>
            </menus>
        </MenuBar>
        <StackPane fx:id="openGLPane" VBox.vgrow="ALWAYS" />
    </children>
</VBox>

Ok I figured out some things:

I changed the initial block of code that sets up the window and NewtCanvasJFX to listen for width/height properties from the stackpane and forward them to Window which makes resizing work but it acts as if the initialize size is the minimum (see below).  The next question I have is, with NewtCanvasJFX can other JavaFX elements such as Slider be drawn on top of it?  By "on top" I mean z-order or overlayed. Under normal circumstances in JavaFX it is possible to have Sliders and buttons on top of other things using StackPane.

      glWindow = GLWindow.create(capabilities);
      glWindow.setSize(300, 250);
      this.glPanel = new NewtCanvasJFX(glWindow);
      this.glPanel.setWidth(300);
      this.glPanel.setHeight(250);
      openGLPane.getChildren().add(0, glPanel);

      glWindow.addGLEventListener(this);

      this.openGLPane.widthProperty().addListener(new ChangeListener<Number>()
      {
        @Override
        public void changed(ObservableValue<? extends Number> observable, Number oldValue, Number newValue)
        {
          System.out.println("resizing");
          glWindow.setSize(newValue.intValue(), glWindow.getHeight());
        }
      });

      this.openGLPane.heightProperty().addListener(new ChangeListener<Number>()
      {
        @Override
        public void changed(ObservableValue<? extends Number> observable, Number oldValue, Number newValue)
        {
          System.out.println("resizing");
          glWindow.setSize(glWindow.getWidth(), newValue.intValue());
        }
      });

I would refrain from it, as this is more intended
to be a vehicle to have an opque child window
exposing 'a main scene' and doesn't integrate too
well into JFX.

For the latter a shared native image buffer would be desired,
to be rendered into offscreen via FBO.
AFAIK there was something implemented lately in JFX,
have a look at the bug reports.

If you like make your NewtCanvasJFX findings persistent
in our bugzilla, PM me so I can give you an account.

Cheers, Sven

On 1/10/20 6:06 PM, MikeORC [via jogamp] wrote:

Yes the new JFX thing that was implemented is the WritableImage class now takes a PixelBuffer as a constructor argument.
So a ByteBuffer can be allocated, given to a PixelBuffer then put into a WritableImage, however it is still necessary to glReadPixels the framebuffer into the ByteBuffer.  The improvement is that one less copy is necessary in JavaFX 13 where before there was an additional copy that had to be made after the glReadPixels the data had to be copied into the WritableImage.


JavaFX 13 Version:
https://openjfx.io/javadoc/13/javafx.graphics/javafx/scene/image/WritableImage.html

JavaFX 11 Version:
https://openjfx.io/javadoc/11/javafx.graphics/javafx/scene/image/WritableImage.html

In 2015 I elaborated options
<https://jogamp.org/bugzilla//show_bug.cgi?id=607#c20>

Especially for a low memory footprint, high performance and
reducing library dependencies - I would recommend to drop
JavaFX's current GL binding w/ JOGL to be used by Prism
- and have NEWT replace Glass.

Embedded systems would benefit big time.

+++

WriteableImage is a poor design choice,
as it would have been easy to use something EGL based
for both, GL and Vulkan.

Exposing a factory interface implemented by the user,
who then would provide a matching framebuffer
within OpenJFX's requirements.

<https://jogamp.org/bugzilla//show_bug.cgi?id=607#c27>

That would have allowed a potential zero CPU copy
until the compositor does its thing - compositing ;-)

Exposed this comment here
<https://jogamp.org/wiki/index.php?title=SW_Tracking_Report_Feature_Objectives_Overview#OpenJFX>

I personally prefer a GraphUI/NEWT solution
<https://jogamp.org/wiki/index.php?title=SW_Tracking_Report_Feature_Objectives_Overview#Graph>

+++

It all depends on financing from commercial parties,
as we have surely many ideas to work on.

So again - if any companies are listening and can utilize JogAmp
more for their product, please see
<https://jogamp.org/wiki/index.php/Maintainer_and_Contacts#Commercial_Support>

As of today our project funding has reached ZERO.

;-)

Cheers, Sven

On 1/13/20 6:59 PM, MikeORC [via jogamp] wrote: health & wealth
mailto:[hidden email] ; http://jausoft.com
land : +49 (471) 4707742 ; fax : +49 (471) 4707741
Timezone CET: PST+9, EST+6, UTC+1