Connection between -Xmx and GL_OUT_OF_MEMORY?

Hi,

I've encountered several weird cases of calls to glTexImage3d where I get GL_OUT_OF_MEMORY exceptions when allocating GPU-memory for a texture. Note that I do not upload any actual pixel data ( glTexImage3D(..,..,.., null ) ).
In my test-case I'm trying to allocate ~100MB of GPU memory, which should be perfectly fine with 2GB memory available and GPU-Z reporting only ~70MB used memory.

Somehow I tracked the problem down to the Java Heap size. With "-Xmx1000M" I assigned 1000MB max heap memory to the Java-vm. When I completely delete that flag however, the GL_OUT_OF_MEMORY exceptions do NOT occur and the texture space is allocated just fine.

Do any of you guys have a clue how these two things could be connected? Since I'm only allocating GPU-memory and do not upload any pixels, one thing should not affect the other IMO.

Thanks for any hints ;)

Hi

Some graphics cards silently use a cache on the CPU side for textures and even for VBOs, you cannot be sure that OpenGL allocates some memory really resident on the GPU. This behaviour is not the same on all graphics cards, some of them simply return an OpenGL error when they cannot perform an allocation on the GPU, this is particularly true for VBOs.

The last argument of glTexImage3D is a NIO buffer. It can't be null. This buffer might reside in CPU memory even though you use glMapBuffer (behaviour reproducible with NVIDIA Quadro FX cards).

Edit.: avoid using indirect NIO buffers in such situations of course.

When you call glTexImage3D(), even if the texture will eventually be allocated in video memory on your graphics card, the GL driver is still free to allocate host memory for its own internal use (either temporarily or permanently). If your -Xmx is set too high, the Java VM could simply be taking up so much host memory that the GL driver doesn't have enough left.