GLU tesselation problem

Hi,

we are using the GLUtesselator to triangulate triangles where some area has been cut off.
This works most of the time, but for some cases, the tesselator generator strange results.

I was able to reproduce the issue to a simplified example, where we cut away the upper part of a triangle.
The Java code is this:

// define the triangle corner vertices
double[] vertex0 = new double[] {-3.5355339059327386, -3.5355339059327373, 0.0};
double[] vertex1 = new double[] {-3.8650522668136853, -3.1719664208182263, 0};
double[] vertex2 = new double[] {-3.865052266813685, 3.1719664208182277, 0};

// define the vertices splitting the triangle in two parts
double[] vertex3 = new double[] {-3.7092235489843537, 4.440892098500626E-16, 0.0};
double[] vertex4 = new double[] {-3.8650522668136853, 0.0, 0.0};

GLUtessellatorCallbackAdapter callback = new GLUtessellatorCallbackAdapter() {
  @Override
  public void begin(int aType) {
    System.out.println("type = " + aType);
  }

  @Override
  public void vertex(Object aVertex) {
    System.out.println("vertex = " + aVertex);
  }
};

GLUtessellator tesselator = GLU.gluNewTess();
GLU.gluTessProperty(tesselator, GLU.GLU_TESS_WINDING_RULE, GLU.GLU_TESS_WINDING_POSITIVE);
GLU.gluTessCallback(tesselator, GLU.GLU_TESS_BEGIN, callback);
GLU.gluTessCallback(tesselator, GLU.GLU_VERTEX, callback);

GLU.gluTessBeginPolygon(tesselator, null);

// triangle outer contour (clockwise)
GLU.gluTessBeginContour(tesselator);
GLU.gluTessVertex(tesselator, vertex0, 0, "V0");
GLU.gluTessVertex(tesselator, vertex1, 0, "V1");
GLU.gluTessVertex(tesselator, vertex2, 0, "V2");
GLU.gluTessEndContour(tesselator);

// part of the triangle we want to cut away (define the contour anti-clockwise)
GLU.gluTessBeginContour(tesselator);
GLU.gluTessVertex(tesselator, vertex3, 0, "V3");
GLU.gluTessVertex(tesselator, vertex2, 0, "V2");
GLU.gluTessVertex(tesselator, vertex4, 0, "V4");
GLU.gluTessEndContour(tesselator);

GLU.gluTessEndPolygon(tesselator);

When I run this code, the callback function gives this output:
type = 6           --> GL_TRIANGLE_FAN
vertex = V0
vertex = V1
vertex = V3
vertex = V2

This gives the following 2 triangles: V0-V1-V3; V0-V3-V2. One triangle is missing here in the result (V1-V4-V3).
When I run this sample code on cases where it does tesselate properly, the resulting triangles are V0-V1-V4, V0-V4-V3 and V0-V3-V2.

Any idea what's going wrong here?

Maarten

Hello

I advise you to fill a bug report. By the way, a clean JUnit test would help in quickly clarifying the expected results compared to the current ones.