Find position in 3D model from position on 2D screen

Hi,
i'm just starting with jogl and i have a problem:

i have a ground plane (something like a chess board) and i'm showing a view from diagonally above in an angle of roundabout 30° in a JFrame (like the view was in e.g. age of empires).
Now i've added a MouseListener that returns me the position of the mouse on my screen on clicking. How do i find out which square i clicked?
The MouseListener gives me x and y on the 2D screen. How do i convert them into the coordinates of the underlying square on the ground plane?

Sorry if i explained in a complicated way or ask something easy. Any help very welcome!

I would recommend using the JOGL technique called Picking. Even tough is a bit slow, I think it would suit your application.

Here is a tutorial: http://fivedots.coe.psu.ac.th/~ad/jg2/, http://fivedots.coe.psu.ac.th/~ad/jg2/ch17/index.html.

Hi

There is an example based on JOGL here:
http://forum.jogamp.org/2D-Picking-with-Jogl-tp2594712p2601612.html

The use of hardware accelerated OpenGL picking is highly discouraged:
http://www.opengl.org/wiki/Common_Mistakes#Selection_and_Picking_and_Feedback_Mode

Color picking might not work on displays that "simulates" some parts of the palette.

The most reliable solution consists in using software picking with a third party library. This is a common feature in most scenegraphs including Ardor3D.

Best regards.

Thank you. That's exactly, what i was looking for.
But it does not yet work for me.
It only returns any hits at all, if i use a picking volume of 500x500 in gluPickMatrix (in an 1006x560 viewport).
Code is more or less copy and paste from the tutorial you posted with two differences:

Since BufferUtil does not work anymore, i replaced it with newDirectIntBuffer
                selectBuffer = GLBuffers.newDirectIntBuffer(BUFSIZE);
                // selectBuffer = BufferUtil.newIntBuffer(BUFSIZE);

And i added an additional argument to glGetIntegerv since i demanded one:
                gl2.glGetIntegerv(GL2.GL_VIEWPORT, viewport, 0);
                // gl2.glGetIntegerv(GL2.GL_VIEWPORT, viewport);

Could this be a my problem?

Don't know Ardor3D, but thank you for your input.
I can only read your examples and google Ardor3D later, because i have to go to bed now. Have to work tomorrow :(

I will reply again tomorrow, when i tried it.

On 06/26/2012 08:12 PM, nabo [via jogamp] wrote:
> Hi,
> i'm just starting with jogl and i have a problem:
>
> i have a ground plane (something like a chess board) and i'm showing a view
> from diagonally above in an angle of roundabout 30° in a JFrame (like the view
> was in e.g. age of empires).
> Now i've added a MouseListener that returns me the position of the mouse on my
> screen on clicking. How do i find out which square i clicked?

May I refer to our previous discussion about mouse selection / picking ?

<http://forum.jogamp.org/Mouse-selection-td3787468.html#a3789778>

Would be best to search the forum first, IMHO.

~Sven

Sorry, i didn't think of it as a selection, but as finding coordinates in the 3D-Modell, so i had no key word to search for

i tried your advice, but it does always return 0 hits. Can you help me with my code? I cannot find the mistake:

        private static final int BUFSIZE = 512;
        private IntBuffer       selectBuffer;

        private void pickModels(GL2 gl2, int x, int y)
        {
                startPicking(gl2, x, y);
                for (Objekt3d objekt3d : objekt3dListe)
                {
                        gl2.glPushName(1 + objekt3dListe.indexOf(objekt3d));
                        objekt3d.draw(gl2);
                        gl2.glPopName();
                }
                selectionMode = false;
                endPicking(gl2);
         }

        private void startPicking(GL2 gl2, int x, int y)
        {
                selectBuffer = GLBuffers.newDirectIntBuffer(BUFSIZE);
                gl2.glSelectBuffer(BUFSIZE, selectBuffer);
                gl2.glRenderMode(GL2.GL_SELECT);
                gl2.glInitNames();
                gl2.glMatrixMode(GL2.GL_PROJECTION);
                gl2.glPushMatrix();
                gl2.glLoadIdentity();
                int viewport[] = new int[4];
                gl2.glGetIntegerv(GL2.GL_VIEWPORT, viewport, 0);
                glu.gluPickMatrix((double) x, (double) (viewport[3] - y), 5, 5, viewport, 0);
                System.out.println(x + " " + ((viewport[3] - y)));
                float widthHeightRatio = (float) drawable.getWidth() / (float) drawable.getHeight();
                glu.gluPerspective(30.0, widthHeightRatio, 1.0, 1000.0);
                gl2.glMatrixMode(GL2.GL_MODELVIEW);
        }

        private void endPicking(GL2 gl2)
        {
                gl2.glMatrixMode(GL2.GL_PROJECTION);
                gl2.glPopMatrix();
                gl2.glMatrixMode(GL2.GL_MODELVIEW);
                gl2.glFlush();
                int numHits = gl2.glRenderMode(GL2.GL_RENDER);
                System.out.println("endpicking, hits: " + numHits);
                processHits(numHits);
                selectionMode = false;
        }

If Picking is giving you trouble, you might want to do what I do. Simply convert you mouse's X,Y coordinates to Opengl's X,Z coordinates, and done!

What do you mean by 'convert'? Is this what gluUnproject does?

What I do in Android is this, maybe it works for your project. I get the current pointer (mouse arrow, finger) X Y coordinate which are for example X = 25, Y = 40.563. Then I convert that to X = 25, Z = 40.563. And Done!

I really don't know what else to tell you. Hope this helps.