high memory consumption (IntIntHashMap)

Hi!

45% of my memory is occupied by IntIntHashMap instances and their entries, they are created by JOGL. Has someone noticed this problem? I use JOGL 2.0 beta build 273, GLCanvas with the auto swap buffer mode on (default value).

  does it occur with the latest build too?

as always please provide a little bit more information. e.g some kind of
testcase, heapdump or call stack of the Object instances.

do they go away if you do System.gc() (e.g via VisualVm)? or do they
persist?

best regards,

-michael

On 03/03/2011 05:29 PM, gouessej [via jogamp] wrote:

Hi!

They go away if I do System.gc(). I'm going to switch to a more recent version and take a screen capture of what I get with JVisualVM.

I can use TPTP to find what is wrong if necessary.

On Friday, March 04, 2011 10:50:00 gouessej [via jogamp] wrote:
>
> Hi!
>
> They go away if I do System.gc(). I'm going to switch to a more recent
> version and take a screen capture of what I get with JVisualVM.
>
> I can use TPTP to find what is wrong if necessary.

Awesome Julien.

This is the 2nd performance issue you found.
(First one AWT/GLCanvas + AutoSwap-Off)

We shall take care of that for rc3 for sure.

Thank you.

~Sven

A collegue is going to help me with YourKit and I'm going to give a try to TPTP. I use the build 293 and I will use the build 341 when I can use 7zip (nice choice :)).

Sven Gothel wrote

This is the 2nd performance issue you found.
(First one AWT/GLCanvas + AutoSwap-Off)

This is the third one if you count the high memory consumption in the TextRenderer.

  i monitored a few demos (jocl-jogl interoperability stuff and a VBO
heavy project) and wasn't able to reproduce it. (IntIntHashMap$Entry
never went above 1-2% of the heap).

would be great if you could provide call stacks of the allocated
instances. I am sure both YourKit and TPTP should be able to generate
them (NetBeans Profiler works also).

thanks,
michael


On 03/04/2011 03:42 PM, gouessej [via jogamp] wrote:

I'm investigating, it seems to come from GLStateTracker:


The pushAttrib method seems to allocate a lot of things.

On Friday, March 04, 2011 17:14:15 gouessej [via jogamp] wrote:
>
> I'm investigating, it seems to come from GLStateTracker:
> http://forum.jogamp.org/file/n2634263/highMemoryConsumption_IntIntHashMap.png 
>

Please review this change .. (the more the merrier)
  http://jogamp.org/git/?p=jogl.git;a=commitdiff;h=a167589233db343ac22b761d30d9dcd73016fe54

Inspecting the code, I realized that popAttrib() even has overwritten the current
client pixel-store state if it's corresponding pushAttrib() didn't happen with CLIENT_PIXEL_..
Which was IMHO a semantical bug .. and should be visible in applications push/pop a lot
with different bitmasks, where the pop would errorneous overwrite the pixel-store state.

This also removes the unnecessary IntIntHashMap creation on popAttrib() at all,
if this was the case of your test case memory peak.

Would be actually nice to have your test case at hand ..

However, pls review my code change - thank you.

~Sven

Hi!

I'm not at work today. I'll test the latest build Monday morning. My test case is huge and uses a lot of glEnable, glDisable, glPushAttrib, glPushClientAttrib, etc... I will try to write a smaller test case if the bug persists. I have to be more precise, I use the debug pipeline.

On 03/05/2011 03:33 AM, Sven Gothel [via jogamp] wrote:

On Friday, March 04, 2011 17:14:15 gouessej [via jogamp] wrote:
>
> I'm investigating, it seems to come from GLStateTracker:
> http://forum.jogamp.org/file/n2634263/highMemoryConsumption_IntIntHashMap.png 
>

Please review this change .. (the more the merrier)
  http://jogamp.org/git/?p=jogl.git;a=commitdiff;h=a167589233db343ac22b761d30d9dcd73016fe54

looks good, I added a few things:
https://github.com/mbien/jogl/commit/0b57c668703de3717badf8e779a91e64773d7f61
commit message:

 - using LinkedList as stack replacement for non index based push/pop
 - increased initial map size since the default values already exceeds the default mapsize
 - size()==0 -> isEmpty() (can be significantly faster if the deque impl would change in future)
 - map copy with initial size
 - minor other changes

-michael

Inspecting the code, I realized that popAttrib() even has overwritten the current
client pixel-store state if it's corresponding pushAttrib() didn't happen with CLIENT_PIXEL_..
Which was IMHO a semantical bug .. and should be visible in applications push/pop a lot
with different bitmasks, where the pop would errorneous overwrite the pixel-store state.

This also removes the unnecessary IntIntHashMap creation on popAttrib() at all,
if this was the case of your test case memory peak.

Would be actually nice to have your test case at hand ..

However, pls review my code change - thank you.

~Sven

---

If you reply to this email, your message will be added to the discussion below:

http://forum.jogamp.org/high-memory-consumption-IntIntHashMap-tp2626543p2636570.html

To start a new topic under jogamp, email [hidden email]
To unsubscribe from jogamp, click here.

Nice foundings :) Thanks.

The high memory consumption is still there despite your fixes, I'm investigating.

The high memory consumption is still there despite your fixes, I'm investigating.

Is there a way of using the debug pipeline except the GL state tracker?

Hi

Could the API be modified to allow to disable the GL state stracker? I do not succeed in fixing this bug cleanly :(

On Tuesday, March 29, 2011 10:58:17 am gouessej [via jogamp] wrote:
>
> Hi
>
> Could the API be modified to allow to disable the GL state stracker? I do
> not succeed in fixing this bug cleanly :(

Please send/offer a test case, so we can have an opinion - please.

~Sven

I believe this is allocating lots of memory because of TextRenderer calling glPushClientAttrib() and glPopClientAttrib(), maybe twice each in fact.

On my program, I render 50-80 per frame maybe and it allocates 20MB per second at 20 FPS, which is crazy.

It calls these as it does VBO rendering. If you adapt the code and take these calls out my RAM usage goes down to < 1 MB per second, which will just be general other random objects being allocated.

The push/pop methods appear to be called only to restore the VBO state, however, the TextRenderer class unbinds its VBO after use each time, so its merely called to maintain some vertex / tex coord pointers, which generally you reset when you bind your own VBO anyway, and generally you wouldn't have an external VBO bound when calling the text renderer drawing anyway.

I see why its there for general purpose but its a killer on memory allocation, esp for so few calls even per frame 20MB is pretty wild. Also, it still push/pops even if you disable VBO usage with setUseVertexArrays(false) which doesn't seem like its even needed after that is set to false.

Additionally it pushes all attrib bits, does it need to push the pixel store bit along with the vertex array bit, I'm unsure but doubt it. I tested disabling each actually and pixel store bit accounts for maybe 15/20 mb of the allocation and the vertex array bit 5mb per second.

Anyway, hope this helps if people stumble across it. I'd suggest take an adapt the code to comment those out if you get similar and understand it may change some VBO states, but it does unbind the VBO it uses (and therefore maybe yours if you had one set) and ideally you could use it to create your own cached images of the font strings and use them as image renders, or code your own manually is even better. With these calls removed its quite usable and does render quickly as it does a good caching job of generic strings in general, and perhaps the poor performance of glPushClientAttrib / glPopClientAttrib is implementation specific, but these things are nowadays deprecated/avoided with handling your own states, and appear to be poor here.

Another thing to note is that TextRenderer uses TextureRenderer which has some glPushAttrib/glPopAttrib calls, they're not severe on the memory allocation but also not so optimal as your own state handling so again, something to watch for high performance.

Hope this helps!

Hi

Thank you for your suggestions. I forced the text renderer NOT to use VBOs in order to work around an unfixed bugs in the alpha version of my game and after that, I used another approach for text rendering. Resetting the VBO should work most of the time but after one minute of execution, it caused a crash with ARB VBOs (very early implementation, OpenGL 1.3).

You're right, some things could be done to reduce its memory footprint.

There is another text renderer in JOGL but it requires shaders.