Posted by lukej on May 03, 2012; 3:05am
URL: https://forum.jogamp.org/Problems-with-gluUnProject-tp3954975p3957991.html

That is correct, I corrected my statement about a*b vs. b*a in my edit of my second post.  I was getting mixed up with the multiplication order for the OpenGL array order.  FloatUtil.multMatrixf() DOES appear to multiply in the correct order for OpenGL matrices.

However, I am fairly positive that the gluUnProject call is still calling FloatUtil.multMatrixf() with the wrong order.  The results of the recommended code from OpenGL (linked above, I ported to Java) match the results that I was getting from JOGL 1.1.  They do not match JOGL 2.0 (latest build).  I gave examples above of what the float arrays looked like.  Perhaps you could tell me if they are in the correct order or not.  They look accurate to me.  

I am not flipping the matrix around at all.  I am just calling OpenGL to get the modelview matrix, the projection matrix, and the viewport and then passing them into gluUnProject.

For further proof that there seems to be an issue, calling gluUnProject and then passing the results into gluProject does not get me back to where I started (unless one of the matrices is the identity matrix).

If you look at the example code  I linked above here is the line that does the multiplication:
MultiplyMatrices4by4OpenGL_FLOAT(A, projection, modelview);

Note that the GLU code in JOGL 2.0 does the opposite. (modelMatrix, projMatrix, matrixBuf)