NEWT Display's Thread Identity / Lifecycle

NEWT Display's Thread Identity / Lifecycle

Here is a transcript of our discussion on

    url: conference.jabber.org
    room: jogamp

In short .. we may have to redefine the identity / lifecycle
of the Display instance.

(07:58:35 PM) jau: your discussion it is
(07:58:45 PM) snmvaughan: I've been trying to figure out the intended use of the Display reference count.
(07:58:49 PM) jau: ok
(07:58:59 PM) snmvaughan: It apparently is just a count of the number of calls to Display.create.
(07:59:24 PM) jau: the idea is to have _one_ display per thread [ and unique X11 display reference ]
(07:59:40 PM) snmvaughan: Since it is possible to reuse a Display with multiple Screen instances, I don't see a good way to manage the resources (especially the EDT thread).
(07:59:48 PM) jau: under most OS (ex macosx) .. the display per thread can handle input
(08:00:18 PM) jau: 1- the thread creating the window (and the display) shall be the same
(08:00:29 PM) jau: 2- only that thread can deal with input
(08:01:12 PM) jau: so we create the EDT for this display, so the user thread is not disturbed
(08:01:23 PM) jau: the NEWT EDT handles:
(08:01:25 PM) jau: - input
(08:01:29 PM) jau: - window lifecycle
(08:01:55 PM) jau: resource management .. well, it must be done explicitly at a defined time
(08:02:12 PM) snmvaughan: The threads used to create and destroy the Display are different.
(08:02:30 PM) jau: otherwise .. you have no control about the lifecycles of all resources (window, GL, ..) which depend ..
(08:02:55 PM) jau: they might be different, yes
(08:03:05 PM) jau: but EDT does window create and destruction
(08:03:29 PM) snmvaughan: It is easy to call Display.create indirectly.  This means that it is easy to leave the EDT thread for the Display running.
(08:03:31 PM) jau: which is the job for EDT, plus input management, pipe it to the queue
(08:04:04 PM) jau: if you create a Display, you should know what you are doing
(08:04:27 PM) jau: we could enhance it with a window ref counter to block EDT then .. -> low CPU
(08:04:32 PM) jau: good thinking
(08:04:34 PM) snmvaughan: Understood, but using NewtCanvasAWT which reparents the window calls Display.create.
(08:04:46 PM) jau: yep
(08:05:04 PM) jau: (sure there is room for enhancements ..)
(08:05:15 PM) jau: let me look at the junit test case :)
(08:06:02 PM) snmvaughan: ... and that gets back to my questions.  I'd be happy to look into working through some of these issues, but I'm really having trouble with the use of the refCount in Display for the number of calls to Display.create.
(08:06:57 PM) snmvaughan: I would think it would be easier to manage if the Screen registered with the Display, and the Display didn't allocate its native parts, start the EDT, etc. until the it was actually needed.
(08:07:04 PM) jau: so refCount is one pattern for singularity here ..
(08:07:11 PM) snmvaughan: Destroying a screen would then unregister with the display.
(08:07:29 PM) jau: well .. that's just one less command
(08:07:57 PM) jau: you could bloat that up to dpy -> scr -> win
(08:08:00 PM) jau: ref counting ..
(08:08:01 PM) snmvaughan: Leaving the count the way it is now, it would seem that the NewtCanvasAWT would need to basically do something similar to the deep destroy during reparenting.
(08:08:46 PM) snmvaughan: I'm just brainstorming.  As long as I have a working solution I'm happy.
(08:09:32 PM) snmvaughan: We are trying to use NEWT in a multi-window setup.  Since the lives of the windows are independent, we are having trouble shutting down cleanly.
(08:10:17 PM) snmvaughan: The ref counts keep getting off, since the reparenting keeps bumping the refcount while nothing calls destroy.
(08:10:32 PM) jau: yep ..
(08:10:43 PM) jau: just read the code again .. it's just a bug
(08:10:48 PM) snmvaughan: I'm sure we can work around it, but I would think we aren't the only ones running into the problem.
(08:11:10 PM) jau: the !add case in awt/NewtCanvasAWT.java #198 should handle that
(08:11:43 PM) jau: a different pattern .. as you suggested would be:
(08:11:52 PM) jau: - create a Display
(08:12:00 PM) jau: - add a screen - window
(08:12:14 PM) jau: - enable it for destruction if screen count == 0
(08:12:26 PM) jau: same for screen / window :)
(08:12:36 PM) jau: would that .. be convenient ?
(08:12:48 PM) snmvaughan: I think I'm missing something.
(08:12:54 PM) snmvaughan: As I understand the usage:
(08:13:18 PM) snmvaughan: - Create a Window, which creates a Screen and Display either directly or indirectly.
(08:13:33 PM) jau: well .. the basic code is:
(08:13:33 PM) snmvaughan: - Create the NewtCanvasAWT passing in the child.
(08:14:03 PM) jau: lets keep NewtCanvasAWT out of this .. for easyness
(08:14:05 PM) snmvaughan: - Make it visible, which discards the Screen/Display and reparents the window.  This ends up being the same Display.
(08:14:11 PM) jau: (and it doesnt make a diff)
(08:14:32 PM) jau: :)
(08:14:48 PM) snmvaughan: Ok.  Once you are done.  call destroy(true) on the window, but the EDT is left running.
(08:14:58 PM) jau: it's special NewtCanvasAWT code ..
(08:15:09 PM) jau: destroy on which window ?
(08:15:14 PM) jau: that's the Q ..
(08:15:37 PM) jau: you have 2 ..
(08:15:43 PM) jau: GLWindow
(08:16:25 PM) snmvaughan: GLWindow wraps the actual Window.  I don't think that contributes to the problem.
(08:16:28 PM) jau: NewtCanvasAWT is just the AWT glue / connector
(08:16:55 PM) snmvaughan: Is there a better mechanism for incorporating NEWT into a larger app?
(08:17:26 PM) jau: before you dismiss our AWT/NEWT completely .. let's understand the problem 1st :)
(08:17:46 PM) jau: we made it to solve the AWT integration
(08:17:53 PM) jau: and swing 'overlay'
(08:18:02 PM) jau: without bothering with the AWT EDT ..
(08:18:06 PM) jau: back to the problem
(08:18:18 PM) jau: the phenomenom should be:
(08:18:25 PM) snmvaughan: Understood.  That is why I'm so interested in its use.
(08:18:26 PM) jau: glWindow.destroy(true)
(08:18:45 PM) jau: should destroy the whole thing ..
(08:19:09 PM) snmvaughan: It doesn't, because the refcount is 2 before the call to glWindow.destroy(true).
(08:19:09 PM) jau: (if you know the ref counts)
(08:19:25 PM) jau: (only destroys the window .. right)
(08:19:41 PM) jau: ok .. let's declare this behavior a bug .. and create a junit test case
(08:20:07 PM) jau: happy to solve it .. indeed .. the whole windowing thing is quite confusing,
(08:20:23 PM) jau: but to make it happen for all platform, the abstractions are necessary
(08:21:19 PM) jau: so your test case would be like: TestParenting01cAWT.testWindowParenting01CreateVisibleDestroy1()
(08:21:22 PM) snmvaughan: I modified my copy of TestParenting01cAWT.testWindowParenting01CreateVisibleDestroy1
(08:21:28 PM) jau: :)
(08:21:33 PM) jau: same thought ..
(08:21:39 PM) snmvaughan: EDTUtil edt1 = glWindow1.getScreen().getDisplay().getEDTUtil();
        Assert.assertTrue (edt1.isRunning());
       
        glWindow1.destroy(true);
        Assert.assertEquals(true, glWindow1.isDestroyed());
       
        edt1.waitUntilStopped();
        Assert.assertFalse ("EDT has not stopped", edt1.isRunning());
(08:22:20 PM) snmvaughan: The waitUntilStopped doesn't return (or times out depending on how you run JUnit).
(08:22:20 PM) jau: thats a start .. great
(08:22:53 PM) jau: wait until stopped does only that :)
(08:22:58 PM) jau: not trying to stop it
(08:23:15 PM) jau: look in Display line 228
(08:23:21 PM) snmvaughan: Right.
(08:23:30 PM) jau: releaseEDTUtil()
(08:23:54 PM) snmvaughan: I had been looking into switching things around so that garbage collection, closing screens, etc. would actually adjust the refcount.
(08:23:59 PM) jau: so an explicit destroy is needed .. and you say .. refCount was 2, so that makes it 1 .. and still running ?!
(08:24:04 PM) snmvaughan: Right.
(08:24:15 PM) jau: now we have a test case sir :)
(08:24:33 PM) jau: I will take care of it .. since I am currently dealing with it anyways (macosx)
(08:24:34 PM) snmvaughan: Making the NewtCanvasAWT visible actually increments the refcount (assuming that the original Display didn't have a name).
(08:25:05 PM) jau: the creation .. yes
(08:25:59 PM) jau: the problem here is the lazy creation of the NEWT / NativeWindow resources,
(08:26:16 PM) jau: which are necessary to attach the window to the right resources ..
(08:26:28 PM) snmvaughan: I also ran into a bug which made calling setNEWTChild not function once the screen was visible if it didn't have a child to begin with.
(08:26:39 PM) jau: ie Display/Screen .. etc plus having the same attributes (visual id, ..)
(08:26:57 PM) jau: so can you help me out here with:
(08:27:00 PM) jau: fork my repo
(08:27:26 PM) snmvaughan: The reason I originally asked about the expected use was it seems to me that the problem comes down to who is responsible for managing the Display.
(08:27:36 PM) jau: add those test cases in a new class (TestNEWTDisplayRefCount...
(08:27:45 PM) jau: and I will pull .. and work on it ..
(08:27:49 PM) snmvaughan: The NewtCanvasAWT is manipulating the display but it doesn't destroy any existing display.
(08:28:02 PM) jau: which is NewtCanvasAWT fault .. sure
(08:28:23 PM) jau: indeed responsibilities are mixed here .. a design / code bug
(08:28:39 PM) jau: usually the user is responsible for NEWT resources
(08:28:39 PM) snmvaughan: But that assumes that the Display is always created when building a window.  It is easy to create a situation with 1 display and 2 screens with a refcount of 1.
(08:29:21 PM) jau: sure .. the 'normal' case
(08:29:23 PM) snmvaughan: So we are assuming that it would really be 2 displays and 2 screens, with potentially the 2 displays being the same?
(08:29:40 PM) jau: nope
(08:29:57 PM) jau: the simple case is: one thread - one Display
(08:30:12 PM) jau: that dpy can have whatever number of screens and windows
(08:30:20 PM) jau: the user creates it and destroys it
(08:30:26 PM) jau: complicated case:
(08:30:43 PM) jau: the code doesn't know .. (NewtCanvasAWT, or some other complicated API)
(08:31:05 PM) jau: so .. same thread .. tries to create a display here and there .. with the same 'name'
(08:31:17 PM) jau: resolves in the same Display reference with refCount = 2
(08:31:27 PM) snmvaughan: Agreed.  So if I create a Display/screen/window and keep attaching window, how will I know when to do a deep destroy?
(08:31:45 PM) jau: responsibilities .. user app ..
(08:32:05 PM) jau: we could say TLS thread ..
(08:32:12 PM) snmvaughan: TLS?
(08:32:14 PM) jau: if refCount == 0 .. destroy
(08:32:23 PM) jau: thread local storage (which is being used in Display)
(08:32:35 PM) jau: see the use case contract above ..
(08:32:56 PM) jau: so .. let's say you really cannot say when the dpy can be destroyed ..
(08:33:12 PM) snmvaughan: That was another potential problem.  If I have the Display instance and call destroy, I'm not necessarily on the correct thread for the ThreadLocal.
(08:33:26 PM) jau: the we could use the contract .. ie refCount==0 via window.destroy() -> screen.destroy() -> dpy.destroy()
(08:33:48 PM) jau: well .. that should be ok ..
(08:33:56 PM) jau: oh no .. right
(08:34:07 PM) jau: the TLS is different .. hence you get a new one .. indeed
(08:34:31 PM) snmvaughan: Is the key for the Display just the type and name, or does it have to include the thread?
(08:34:37 PM) jau: that would intro a static-locked storage :(
(08:34:56 PM) jau: thread is implicit .. since it's TLS
(08:35:06 PM) snmvaughan: Just for creation/destruction.
(08:35:15 PM) jau: so if we 'solve' this issue ..
(08:35:18 PM) snmvaughan: I guess I'm asking about the need for TLS.
(08:35:30 PM) jau: we need a static-storage for Display lists
(08:35:50 PM) jau: and the key would be Name+Type+TLS
(08:35:55 PM) jau: brb .. doorbell :)
(08:35:59 PM) snmvaughan: If creation/destruction rarely happens, and all native creation/destruction happens by the EDT of the object, maybe it doesn't matter.
(08:38:05 PM) snmvaughan: A majority of the work is run by separate threads.  Destruction would be requested on the EDT of the display.  locking of the static map would only occur when creating/finding an existing Display or when destruction of the display actually was about to happen.
(08:40:34 PM) jau: Close ..
(08:40:54 PM) jau: differentiate EDT and Usr thread ..
(08:41:09 PM) jau: the EDT thread is 'hidden' attached to the Usr thread
(08:41:13 PM) jau: Usr = User
(08:41:29 PM) jau: the Usr thread is currently the implicit key -> TLS
(08:41:35 PM) jau: (part of it)
(08:41:58 PM) jau: due to reparenting .. it happens that the window switches 'displays'
(08:42:27 PM) jau: nope - damn .. forgot the use case ..
(08:43:52 PM) snmvaughan: This basically means that the user thread for NewtCanvasAWT is always the AWT EDT thread.
(08:44:20 PM) jau: yep .. addNotify etc ..
(08:44:31 PM) jau: only the 1st time .. it's the Usr thread
(08:44:40 PM) snmvaughan: The user thread is only used for creation and destruction calls from the Display point of view.
(08:45:02 PM) snmvaughan: Rendering/animation requests could actually come from a 3rd thread.
(08:45:03 PM) jau: yep .. and of course to render something :)
(08:45:13 PM) jau: or that (Animator or Usr)
(08:45:56 PM) jau: another important detail here:
(08:45:57 PM) snmvaughan: BTW.  I really like NEWT in general. :)
(08:46:05 PM) jau: good :)
(08:46:35 PM) jau: to share OpenGL resources .. (even btw 2 windows .. or framebuffers) they need to be on the same display
(08:46:59 PM) jau: I see where this discussion could lead us :)
(08:47:29 PM) jau: the association of Usr-Thread as part of a Display key is redundant .. maybe wrong
(08:47:46 PM) jau: looks like .. we 'just' need something like:
(08:48:09 PM) jau: - create display explicitly or shared
(08:48:23 PM) jau: if shared - use some refCount mech.
(08:48:50 PM) jau: if explicitly .. destroy or so .. hmm .. guess I have to meditate about this more
(08:49:00 PM) snmvaughan: Ok.
(08:49:10 PM) jau: but it should be more simple .. right
(08:49:41 PM) jau: btw .. finalizer .. garbage collector .. JVM shutdown hooks .. they all are not working reliable for destruction
(08:49:42 PM) snmvaughan: Another thing to remember is that the original GLWindow/WIndow/Screen/Display may have had a name, while the NewtCanvasAWT always uses null (resulting in a name of nil).
(08:50:30 PM) jau: ok .. I copy/paste our writing .. and you send me your pull request for junit tests ?
(08:50:32 PM) snmvaughan: I agree that they can be finicky.  I only mentioned them because I was thinking of fall back positions if a programmer makes a mistake.
(08:51:22 PM) snmvaughan: I don't understand the pull request.  I thought git used push?
(08:51:28 PM) jau: at least the AWT/NEWT event pipelining / adapters are easy and convenient :)
(08:51:46 PM) snmvaughan: :-P new to git.
(08:51:51 PM) jau: your pull request .. so I can pull your pushed code [to your server/repo] :)
(08:52:48 PM) jau: or make the format email thingy (see above .. earlier discussion)
(08:53:16 PM) snmvaughan: Do I use the the repo git://github.com/sgothel/jogl.git?
(08:53:20 PM) jau: and please cc me directly with your email .. thx .. sgothel at jausoft dot com
(08:53:37 PM) jau: pls .. but fork it with your github account
(08:53:48 PM) jau: and pull it (via ssh) from there ..
(08:53:53 PM) jau: so you can push back to your fork
(08:53:58 PM) jau: and I can pull from there ..
(08:54:05 PM) jau: see github help how to fork
(08:54:13 PM) jau: and our FAQ .. etc ..
(08:54:20 PM) snmvaughan: Ok.  I'll take a look.  Thank you.
(08:54:28 PM) jau: or send me an email with a formatted git thingy :)
(08:54:32 PM) jau: thank you
(08:54:57 PM) jau: let's cont this talk to make NEWT better .. very much appreciated