Posted by
Sven Gothel on
Sep 18, 2012; 3:18am
URL: https://forum.jogamp.org/Turning-off-XInitThreads-tp4026119p4026209.html
On 09/17/2012 12:37 PM, gouessej [via jogamp] wrote:
> Sven Gothel wrote
> Well, a quick review showed that it is not possible to easily
> drop the multithreading requirements of XLib, b/c:
> - We reuse display connections in NEWT which
> maybe used from multiple threads. X11 locking is quite fast here,
> since usually xcb is being utilized as the backend.
> Simply removing XInitThreads()
> (even though no explicit XLockDisplay/.. is in place) will cause havoc.
> xcb_ actually detects concurrent access and aborts JOGL in this case.
>
> Why do we reuse display connections in NEWT? Is the creation of such a
> connection very expensive?
Good that you ask .. so allow me to elaborate on this
whole X11 Display / Locking issue .. it's a world of pain :)
Basics:
X11 is network based, using fast local transfers or other slow media.
A X11 display connection (DPY) communicates w/ a device (X server)
which may be local (fast) or remote (slow).
One site can have multiple connection names, e.g. X server etc.
There is a DPY limit, 255 connections are allowed _to_ one X11 server (?)
1 DPY gives you one communication queue to the server,
each is intrinsically thread safe if used by one thread at a time.
If sharing a DPY one would either need to use
[1] XLockDisplay/XUnlockDisplay, which requires XInitThreads to work
[2] Application level locking
Depending on the impl. [1] is quite fast today (XCB backend).
A bad example of [2] is AWT, i.e. using one global lock for the whole
toolkit (on X11). In general it would be a very bad idea to use
one DPY for the whole 'location'. Stuttering and other blocking
artifacts would be experienced .. as you are used w/ the
one-for-all EDT architecture as experienced w/ AWT, Cocoa, SWT, ..
Since the goal already achieved is to use one 'display' w/ multiple
rendering instances w/o blocking, we can either use:
[a] one DPY for each thread and use case
[b] X11 locking
The master branch uses a mix of [a] and [b] already,
where a shift towards [a] happened a while ago in the offscreen
surface and drawable implementation.
Now lets have a view back in history ..
- In the beginning .. JOGL1, followed more the EDT approach
utilizing the ThreadUtil. Of course, at this time AWT and Swing
were the only drawable surfaces available.
You could disable the EDT funneling via properties, they still exist.
However, since AWT is not implemented to be thread safe,
but single threaded - your mileage may vary (YMMV).
- On AWT we lock surface and the so called JAWT global lock.
Hence rendering in multiple AWT surfaces (GLCanvas)
from multiple threads will serve you a stuttering experience.
You will also notice a lag of input, since the global lock
synchronizes both, the input event queue and the surfaces - all resources.
- We created NEWT and hence AWT constraints were no more required to follow.
We were asked which threading model NEWT follows.
The simple answer was none at all, i.e. it should be thread safe.
In the end - due to conveniences, the NEWT EDT semantic was born.
NEWT's EDT intend was to only dispatch events from the native queue.
Nevertheless we were required to also use it to handle
the native windowing lifecycle actions like create / destroy windows.
In the end .. as it is of today, you can use a NEWT surface
to render into from any thread, while locking it's surface and DPY.
Since we used [b], a shared DPY w/ XInitThreads, we are thread-safe
and resource efficient. Still .. having multiple NEWT windows
operating on one shared DPY may reduce some responsiveness.
This is the balance to discuss here.
The current DPY situation regarding locking, open and close ..
- NVidia plays nice w/ open and closing of display connections
- AMD has an open/close order bug, i.e. DPY must be closed in the order
they were opened, hence X11Util funnels all DPY open and close ops.
If using AMD, we never close the DPY, but reuse them (not too bad).
At JVM shutdown we close them in the proper order,
which avoids leaks and a SEGV at JVM shutdown .. you may
have been bothered with it.
- XCB backend introduced another hack, it's locking bug forced us
to have 1 DPY per EDT and one per 'renderer'.
Read: Both make up one NEWT X11 Display, so a NEWT X11 Display
opens _two_ DPY.
++++
The new branch x11_no_xinitthreads, test it!
- One good unit test here demonstrating the difference of
the master branch [b] and the new x11_no_xinitthreads branch [a]
is TestInitConcurrentNEWT, which opens multiple NEWT windows
each in it's own thread and renders them.
With [b] you will notice that it takes a little time
until all windows become established, where with [a]
realization of all windows happens almost at the same time
even though it takes longer until they all show up at once.
The total time delta is almost zero, a few milliseconds
pro [a]. However, resource costs for [a] explodes a little bit.
20 NEWT Windows w/ DPY: [b] == 2 DPYs, [a] == 40 DPYs
Of course, this is not the regular use case ..
As mentioned above, it is to discuss which solution is preferable.
PRO x11_no_xinitthreads branch:
- more clean and less complex implementation
- easier to maintain
- neither requiring not bothering w/ XInitThreads,
nor XLockDisplay/XUnlockDisplay
- no questionable locking parts in implementation
- may solve issues folks had w/ IDEs like Eclipse ?
E.g. Wade and others reported different behavior
when calling JOGL's initialization at a different time.
- multiple NEWT windows running [each in their thread]
on one display run more in a separate fashion.
No interference nor blocking from other same 'display' NEWT windows
PRO master branch (old style):
- it's tested and used today
- uses less DPY resources
- much harder to hit the DPY limit of 255 connections to one server
The educated gut feeling will favor 'x11_no_xinitthreads'.
There has been quite some change in the NEWT X11 Display domain:
NEWT: Changed Lifecycle of Display/Screen
<
http://jogamp.org/git/?p=jogl.git&a=search&h=HEAD&st=commit&s=Changed+Lifecycle+of+Display%2FScreen>
also searching for 'X11 Display' reveals above mentioned change sets:
<
http://jogamp.org/git/?p=jogl.git&a=search&h=HEAD&st=commit&s=X11+Display>
My conclusion so far - if you could test the new branch and:
- it works well
- even fixes maybe a bug (Eclipse, IDEs, .. Wade ?)
- the DPY limit doesn't bother you
- no other concerns ..
then we could move over and clean up accordingly.
Sorry for my very long writeup, but since the matter is quite complex
and important to our X11 port I believe being more verbose and detailed
is hurting less as if being too brief.
Cheers, Sven