Bug 502

Summary:	X11/NEWT Stalling due to libX11/XCB Multithreading bug (libX11 1.4.2, libXCB 1.7; ubuntu 11.04, ..)
Product:	[JogAmp] Newt	Reporter:	Sven Gothel <sgothel>
Component:	x11	Assignee:	Sven Gothel <sgothel>
Status:	VERIFIED FIXED
Severity:	critical
Priority:	---
Version:	1
Hardware:	All
OS:	linux
Type:	---	SCM Refs:	56e46acce6d26e0fdc2fd317649b11f1f71f621a 9ed513e9a9616f6028084df4c650c8caf31ea49d
Workaround:	---
Attachments:	stack traces .. ubuntu 10.10 64bit libX11.so 1.3 and libxcb.so 1.5 ..

Description Sven Gothel 2011-06-14 03:24:54 CEST

As discussed in the libX11/XCB mailinglist and bugzilla
  - https://bugs.freedesktop.org/show_bug.cgi?id=20708
  - http://lists.freedesktop.org/archives/xcb/2011-March/006842.html

there is a stall when accessing OpenGL and X11 in a multithreaded manner (as we do at least with NEWT).

The affected version are
  - libX11 1.4.2
  - libxcb 1.7

The fix seems to be in the libX11 master 
  - fd85aca7a616c595fc17b2520f84316a11e8906f (14 Mar 2011)
and incl. in the upcoming libX11 version 1.4.3

I currently build libX11/libxcb binaries and will verify w/ those later on.

~Sven

Comment 1 Sven Gothel 2011-06-14 03:26:57 CEST

Reproduced on jenkins slave w/ ubuntu 11.04 and my dev machine.

Can only be avoided by a proper bugfix, or extensive X11 Display locking,
which we actually try to avoid to benefit from multithreading - and support it.

Comment 2 Sven Gothel 2011-06-14 03:28:11 CEST

Diff tested locking strategies (avoiding the libX11/xcb bug):

1) OK: no XInitThreads, lock AWT, lock X11

2) NO: no XInitThreads, lock X11 (xcb assert)

3) NO: XInitThreads, no lock (hang ..)

4) OK XInitThreads, lock X11 (hang ..)

Comment 3 Sven Gothel 2011-06-14 03:30:32 CEST

Created attachment 259 [details]
stack traces ..

Comment 4 Sven Gothel 2011-06-14 04:02:38 CEST

Sadly the libX11 'fix' fd85aca7a616c595fc17b2520f84316a11e8906f (14 Mar 2011)
doesn't solve the issue .. as one reviewer already learned (bug was reopened).

Comment 5 Sven Gothel 2011-06-14 13:41:47 CEST

Created attachment 260 [details]
ubuntu 10.10 64bit libX11.so 1.3 and libxcb.so 1.5 ..

ubuntu 10.10 64bit libX11.so 1.3 and libxcb.so 1.5 ..

tested those against ubuntu 11.04 64bit, which removes the stalling bug 502

Comment 6 Sven Gothel 2011-06-14 13:42:39 CEST

.. see attachment 260 [details] - working w/ older libX11/libxcb

Comment 7 Sven Gothel 2011-06-26 08:11:12 CEST

Well, actually it's a workaround not a fix, which would be a patch in libX11/libxcb.

http://jogamp.org/git/?p=jogl.git;a=commit;h=56e46acce6d26e0fdc2fd317649b11f1f71f621a

Woraround for bug 502: X11/NEWT Stalling due to libX11/XCB Multithreading bug (libX11 1.4.2, libXCB 1.7; ubuntu 11.04, ..)

- set the boolean property 'nativewindow.x11.mt-bug' to 'true',
  indicating the erroneous libX11/libXCB behavior. This will enable extensive X11 locking even in NEWT.

Comment 8 Sven Gothel 2011-08-30 00:23:08 CEST

Since libX11/xcb doesn't seem to be fixed anytime soon, 
a true workaround has to be reached w/o 'over locking'.

Comment 9 Sven Gothel 2011-08-30 03:52:40 CEST

It turns out that the race condition is related to the parallel
X11 Display connection usage of GLX/OpenGL and event dispatching.
This workaround utilizes 2 X11 Display handles, one for windowing/OpenGL
and one for event dispatching.
This approach allows us to cont. multithreading use w/o locking the display
and works on both implementations, the old bug-free libX11 and the 'new' buggy one.
    
Downside is the little resource overhead of the 2nd X11 Display connection .. well.