Shader Uniform Sampler2D Update in TextureUnitState not applied

Hello Jogamp Java3D Community,

I am currently trying to implement a "Render To Texture" pass in immediate mode rendering, so I can decide on custom masks, do PostFX and in general have control over multiple shader passes. Right now I am at the stage, where I can use GraphicsContext3D.readRaster(Raster) after GraphicsContext3D.draw(Shape3D) to get an ImageComponent2D of the masks I need from a first shader pass. I can then create a Java3D Texture2D and set the TextureUnitState's Texture of the second pass shader. My problem at this point is tat if I just do Texture2D.setImage(Raster.getImage()) to a already existing Texture2D object and then update the TextureUnitState of the second pass shader with .getTextureUnitState(0).setTexture(Texture2D) (doing this each frame), the Sampler2D uniform in the shader does not seem to update. I can get the Sampler2D uniform to update by creating a new Texture2D object before setting the image. But then another problem arises which is a memory leak.
By debugging I found that the texture Image is indeed updated (in the case of not creating a new Texture2D object every frame) but the TextureUnitState might not communicate with the gpu in that case. Or what are your Ideas? I hope you can help me with this. :)

Here is a small test showing the problem:
(uncommenting the line
tex = new Texture2D(Texture.BASE_LEVEL, Texture.RGBA, width, height); 
in the render method makes it work with a memory leak.
I am using Java3D 1.7.2 from here: https://jogamp.org/deployment/maven-java3d)

```
public class CustomRenderingTest extends JFrame implements Runnable {
    int width = 800;
    int height = 800;
    Raster drawRaster;
    JPanel drawingPanel;
    Canvas3D canvas;
    GraphicsContext3D gc;
    Texture2D tex;
    Shape3D shape;
    Transform3D transformSpin;
    Alpha rotAlpha;


    AppearancePass1 appearancePass1;
    AppearancePass2 appearanceTemp;

    public CustomRenderingTest() {
        initWindow();
        new Thread(this).start();
    }


    @Override
    public void run() {
        gc = canvas.getGraphicsContext3D();
        Sphere s = new Sphere(0.8f);
        GeometryInfo info = new GeometryInfo((GeometryArray) s.getShape().getGeometry());

        BufferedImage bImg = new BufferedImage(width, height, BufferedImage.TYPE_4BYTE_ABGR);
        ImageComponent2D imageBuffer = new ImageComponent2D(ImageComponent.FORMAT_RGBA, bImg, true, true);
        imageBuffer.setCapability(ImageComponent2D.ALLOW_IMAGE_READ | ImageComponent2D.ALLOW_IMAGE_WRITE);
        drawRaster = new Raster(new Point3f(0.0f, 0.0f, 0.0f), Raster.RASTER_COLOR, 0, 0, width, height, imageBuffer, null);
        drawRaster.setCapability(Raster.ALLOW_IMAGE_WRITE | Raster.ALLOW_IMAGE_READ | Raster.ALLOW_SIZE_READ | Raster.ALLOW_SIZE_WRITE);

        appearancePass1 = new AppearancePass1();
        shape = new Shape3D(info.getGeometryArray(), new AppearancePass2(bImg));

        tex = new Texture2D(1,Texture.RGBA, width, height);
        tex.setCapability(Texture.ALLOW_IMAGE_WRITE);

        transformSpin = new Transform3D();
        rotAlpha = new Alpha(-1, 6000);

        while (true) {
            render();
            Thread.yield();
        }
    }

    private void render() {
        // first, apply transforms
        double angle = rotAlpha.value() * 2.0*Math.PI;
        transformSpin.rotY(angle);

        // then render the mask pass
        gc.clear();
        gc.setModelTransform(transformSpin);
        appearanceTemp = (AppearancePass2) shape.getAppearance(); // store the shape's individual appearance for later use (but it must be a individually configured Pass2 appearance)
        shape.setAppearance(appearancePass1);
        gc.draw(shape);

        // after drawing the masks, read framebuffer to drawRaster and update the Pass2 shader's uniform with it
        gc.readRaster(drawRaster);
//        tex = new Texture2D(Texture.BASE_LEVEL, Texture.RGBA, width, height);
        tex.setImage(0, drawRaster.getImage());
        appearanceTemp.updateMaskBufferTexture(tex);

        // clear the screen to draw the second pass
        gc.clear();
        shape.setAppearance(appearanceTemp);
        gc.draw(shape);

        // repeat, if another pass needed (like postFX that need more information than from the mask pass)

        // last action = swap
        canvas.swap();
    }

    private void initWindow() {
        drawingPanel = new JPanel();
        setDefaultCloseOperation(WindowConstants.DISPOSE_ON_CLOSE);
        setTitle("Immediate Multipass Rendering Test");
        drawingPanel.setLayout(new BorderLayout());
        drawingPanel.setPreferredSize(new java.awt.Dimension(width, height));
        getContentPane().add(drawingPanel, java.awt.BorderLayout.CENTER);
        pack();

        GraphicsConfiguration config = SimpleUniverse.getPreferredConfiguration();
        canvas = new Canvas3D(config);
        canvas.stopRenderer();
        SimpleUniverse universe = new SimpleUniverse(canvas);
        universe.getViewingPlatform().setNominalViewingTransform();
        drawingPanel.add(canvas, java.awt.BorderLayout.CENTER);
    }

    public static void main(String args[]) {
        System.setProperty("sun.awt.noerasebackground", "true");
        java.awt.EventQueue.invokeLater(() -> new CustomRenderingTest().setVisible(true));
    }

    /////////////////////////////////////////////////////////////////// SHADER STUFF

    public static class AppearancePass1 extends ShaderAppearance {
        public AppearancePass1() {
            super();
            final Shader vertexShader = loadShader("shaders/Pass1.vert", Shader.SHADING_LANGUAGE_GLSL, Shader.SHADER_TYPE_VERTEX);
            final Shader fragmentShader = loadShader("shaders/Pass1.frag", Shader.SHADING_LANGUAGE_GLSL, Shader.SHADER_TYPE_FRAGMENT);
            final Shader[] shaders = new Shader[]{vertexShader, fragmentShader};
            if (vertexShader == null || fragmentShader == null) throw new RuntimeException("Unable to load Pass1 shader");
            final GLSLShaderProgram glslShaderProgram = new GLSLShaderProgram();
            glslShaderProgram.setShaders(shaders);
            setShaderProgram(glslShaderProgram);
        }
    }

    public static class AppearancePass2 extends ShaderAppearance {
        public AppearancePass2(BufferedImage pass1Masks) {
            super();
            final Shader testShaderVert = loadShader("shaders/Pass2.vert", Shader.SHADING_LANGUAGE_GLSL, Shader.SHADER_TYPE_VERTEX);
            final Shader testShaderFrag = loadShader("shaders/Pass2.frag", Shader.SHADING_LANGUAGE_GLSL, Shader.SHADER_TYPE_FRAGMENT);
            final Shader[] shaders = new Shader[]{testShaderVert, testShaderFrag};
            if (testShaderFrag == null) throw new RuntimeException("Unable to load Pass2 shader");
            final GLSLShaderProgram glslShaderProgram = new GLSLShaderProgram();
            glslShaderProgram.setShaders(shaders);
            setShaderProgram(glslShaderProgram);
            setCapability(Appearance.ALLOW_RENDERING_ATTRIBUTES_WRITE);
            setCapability(ShaderAppearance.ALLOW_TEXTURE_WRITE);

            final GLSLShaderProgram glslShader = new GLSLShaderProgram();
            glslShader.setShaders(shaders);
            setShaderProgram(glslShader);

            TextureLoader loader = new TextureLoader(pass1Masks, TextureLoader.BY_REFERENCE | TextureLoader.Y_UP | TextureLoader.ALLOW_NON_POWER_OF_TWO);
            Texture2D tex = (Texture2D) loader.getTexture();
            ImageComponent2D imageComp = (ImageComponent2D) tex.getImage(0);

            tex.setCapability(Texture.ALLOW_IMAGE_WRITE);
            tex.setBoundaryModeS(Texture.CLAMP);
            tex.setBoundaryModeT(Texture.CLAMP);
            tex.setBoundaryColor(1.0f, 1.0f, 1.0f, 1.0f);
            tex.setImage(0, imageComp);

            TextureAttributes texAttr = new TextureAttributes();
            texAttr.setTextureMode(TextureAttributes.REPLACE);

            TextureUnitState[] tus = new TextureUnitState[1];
            tus[0] = new TextureUnitState();
            tus[0].setCapability(TextureUnitState.ALLOW_STATE_WRITE | TextureUnitState.ALLOW_STATE_READ);
            tus[0].setTexture(tex);
            tus[0].setTextureAttributes(texAttr);
            setTextureUnitState(tus);

            final String[] shaderAttrNames = new String[]{"masks", "screenWidth", "screenHeight"};
            glslShader.setShaderAttrNames(shaderAttrNames);
            ShaderAttributeSet shaderAttributeSet = new ShaderAttributeSet();
            shaderAttributeSet.put(new ShaderAttributeValue("masks", 0)); // TextureUnitState index 0
            shaderAttributeSet.put(new ShaderAttributeValue("screenWidth", 800));
            shaderAttributeSet.put(new ShaderAttributeValue("screenHeight", 800)); // width and height will be updated by buffer every frame, so initialization size is not important.
            setShaderAttributeSet(shaderAttributeSet);
        }

        public void updateMaskBufferTexture(Texture t) {
            TextureUnitState tus = this.getTextureUnitState(0);
            tus.setTexture(t);
            ((ShaderAttributeObject) getShaderAttributeSet().get("screenWidth")).setValue(t.getWidth());
            ((ShaderAttributeObject) getShaderAttributeSet().get("screenHeight")).setValue(t.getHeight());
        }
    }

    public static Shader loadShader(String shaderFileName, final int shaderLanguage, final int shaderType) {
        //... working correctly
    }
}
```

Pass1.vert file content :

```
varying vec3 normal;
void main() {
    normal = vec3(normalize(gl_Normal * gl_NormalMatrix));
    gl_Position = gl_ModelViewProjectionMatrix * gl_Vertex;
}
```

Pass1.frag file content :

```
varying vec3 normal;
void main() {
    gl_FragColor = vec4(vec3(normalize(normal)),1.0);
}
```

Pass2.vert file content :

```
void main() {
    gl_Position = gl_ModelViewProjectionMatrix * gl_Vertex;
}
```

Pass2.frag file content :

```
uniform sampler2D masks;
uniform int screenWidth;
uniform int screenHeight;
void main() {
    vec4 tex = texture2D(masks, vec2(gl_FragCoord.x / screenWidth, gl_FragCoord.y / screenHeight));
    gl_FragColor = tex;
}
```

Hello rndmbt,
and welcome to the forum :)

I think i found a solution utilizing a pool of Texture2D.
CustomRenderingTest.java

I also took the liberty of replacing the while-loop with
a Timer and adding a WindowListener because the program
wasn't terminating for me.
With poolSize=50 memory usage peaks at about 215MB for me.

hope this helps,
basti

Hello basti and thank you for the welcome and fast reply :)

I did some tests but I can't seem to get the texture images to update once the pool reuses the Texture2D Objects starting from index 0. Clearing the pool after one run-through and filling it again also does not trigger garbage collection or prevent the memory leak. So with your approach i get the first 50 textures repeating forever like a small video recording and it does not have a memory leak. But this does not solve my problem as i need to constantly update custom masks - (imagine rendering your own depth or stencil mask in the first pass and i want to use the 4 RGBA channels to have a maximum of 4 masks prepared in the first pass. )

Also thanks for taking the time to send that code

basti, you are so fast I didn't even have the test code working, well done.

I've been digging into this and it looks, on the surface of it, like the original Java3D code doesn't cater for this use. Effectively we just want the TextureRetained to know it's ImageComponent2D has been made "dirty" and for it to call updateTexture2DSubImage on the pipeline when the Shape3D is called to draw.

Here is the only work around I've found so far, as I really haven't been able to find the best/most correct way to do the above properly.

Each Texture2D has it's dirty flags set on construction, but they are marked clean after it has been rendered the first time, so creating a new Textur2D on each render pass will ensure the new raster image data is shown on screen on each draw call.


So this code
getTexture2DFromPool();
tex.setImage(0, drawRaster.getImage());

becomes

//getTexture2DFromPool();
tex = new Texture2D(Texture.BASE_LEVEL, Texture.RGBA, width, height);
tex.setImage(0, drawRaster.getImage());

and your rotating sphere keeps rotating properly.

Of course your texture garage collection of the poorest quality.



I'll keep digging for a real solution, but below are my notes on what needs to be done and why the current code stops it from being done.


The solution is to have TextureRetained.reloadTextureImage called each render pass, if the ImageComponent2D inside it has been updated.

reloadTextureImage is called optionally in TextureRetained.updateNative
if the
boolean reloadTexture = false;
is made true by
if ((resourceCreationMask & cv.canvasBit) == 0)


Now how to get the resourceCreationMask set to match the canvasBit becomes very very complex.


Normally, that is in retained mode it's as trivial as the Texture2D.setImage method calls setImage on the TextureRetained, which fires a texture update message on the queue which eventually via the renderer and a heap of code gets RenderBin.updateCanvasResource(Canvas3D[] canvases) to set the mask.

Immediate mode can't use this path as live/unlive states aren't used by immediate mode on scene side of the scene graph (they are not valid in fact).

I need to find a way to get the mask set, or at least defaulted back to the initial value, so that the texture is reloaded. I think this is somehow related to the Canvas3D and it's lists of used textures, possibly that can be cleared or reset somehow, so it assumes the textures is sees again need loading onto the pipeline.

If you don't like reconstructing the Texture2D constantly, simply creating your own version of TextureRetained  and editing updateNative(Canvas3D cv) with an always true reload might be nicer, but its not a long term solution.

philjord,
thanks, but i guess i have been too fast, not realizing the complexity here ;)

But just for the sake of clarity, what i'm seeing in both cases:

1. constructing new Texture2D each time
tex = new Texture2D(Texture.BASE_LEVEL, Texture.RGBA, width, height);

and 2. reusing existing one
getTexture2DFromPool();

is the same colorful sphere: out256.gif
only the 2nd attempt doesn't produce a memory leak.

@rndmbt and philjord
Should i see something different?
Maybe there's something wrong on my end?

basti

Thanks philjord, that gives me plenty to test out.

I was also digging into the depths of the retained classes but had no success finding where the update of the image is rejected. Did a quick test of changing the updateNative(Canvas3D cv) Method and that indeed is what i was looking for. It makes everything work as intended. So I think i will add a flag to the Texture and TextureRetained classes for now and activate that in the AppearancePass2 Texture.

Hey basti, I will definitely keep your approach in mind because it makes the last n rendered textures available, so I could lookup what has happened there if needed. Though the essential problem was that if an image was already existing in a Texture Object, it was not reloaded in the TextureRetained Object (as far as I understood). Maybe a sphere with object space normals rotating around the y axis is not the best example here because it is in fact repeatable. But imagine a moving camera, controlled by the user, you don't know the position and distance etc. previously. If you set the pool size to 50, then frame 51 would not reload the new image into the texture at index 0 but show the old image of the texture at index 0, as the setting of the image does not work in this immediate mode rendering case.

rndmbt,
now i understand, thanks :)

Something i forgot to mention is that OR-ing capabilities together is not allowed, so they should
be set separately.

imageBuffer.setCapability(ImageComponent2D.ALLOW_IMAGE_READ);
imageBuffer.setCapability(ImageComponent2D.ALLOW_IMAGE_WRITE);
drawRaster.setCapability(Raster.ALLOW_IMAGE_WRITE);
drawRaster.setCapability(Raster.ALLOW_IMAGE_READ);
drawRaster.setCapability(Raster.ALLOW_SIZE_READ);
drawRaster.setCapability(Raster.ALLOW_SIZE_WRITE);

Best regards,
basti